Bug#834968: apt-cacher: Fails to run due to curl errors
Hello Mark, On Sun, Aug 28, 2016 at 10:58:48PM +0100, Mark Hindley wrote: > On Sun, Aug 28, 2016 at 06:06:34PM +0200, Helge Kreutzmann wrote: > > I attached it to this email. > > Thanks, helpful. > > > Sun Aug 28 16:34:53 2016|debug [6473]: Init new libcurl object > > Sun Aug 28 16:34:53 2016|debug [6473]: Setting bandwidth limit to 30 bytes > > This is the cause! > > Have you got a limit configuration option set in > /etc/apt-cacher/apt-cacher.conf conf.d or on the command line in > /etc/default/apt-cacher? Indeed, I just wonder how it got there: root@samd:~# grep limit /etc/default/apt-cacher EXTRAOPT=" daemon_port= limit=30 " Thanks, now apt-cacher appears to work. I guess, best to close this bug. Thanks again for your help. Greetings Helge -- Dr. Helge Kreutzmann deb...@helgefjell.de Dipl.-Phys. http://www.helgefjell.de/debian.php 64bit GNU powered gpg signed mail preferred Help keep free software "libre": http://www.ffii.de/ signature.asc Description: Digital signature
Bug#834968: apt-cacher: Fails to run due to curl errors
On Sun, Aug 28, 2016 at 06:06:34PM +0200, Helge Kreutzmann wrote: > I attached it to this email. Thanks, helpful. > Sun Aug 28 16:34:53 2016|debug [6473]: Init new libcurl object > Sun Aug 28 16:34:53 2016|debug [6473]: Setting bandwidth limit to 30 bytes This is the cause! Have you got a limit configuration option set in /etc/apt-cacher/apt-cacher.conf conf.d or on the command line in /etc/default/apt-cacher? Mark
Bug#834968: apt-cacher: Fails to run due to curl errors
Hello Mark, thanks for your feedback. On Sat, Aug 27, 2016 at 09:55:49AM +0100, Mark Hindley wrote: > On Thu, Aug 25, 2016 at 09:19:14PM +0200, Helge Kreutzmann wrote: > > I try this, but this gives an endless loop of lines: > > Holen:1 http://172.16.18.51:/ftp.de.debian.org/debian testing InRelease > > [251 kB] > > > > (after some minutes, the next same line comes up, I left it running > > for hours without any visual change). > > > > In the error.log I see: > > Thu Aug 25 21:14:23 2016|info [11546]: return_file > > /var/cache/aptproxy/packages/ftp.de.debian.org_debian_dists_testing_InRelease > > aborted by timeout at 6631 of 250532 bytes > > Thu Aug 25 21:14:27 2016|info [11547]: Warning: libcurl failed for > > http://ftp.de.debian.org/debian/dists/testing/InRelease with status Timeout > > was reached > > Thu Aug 25 21:14:28 2016|info [11573]: ALARM! > > /var/cache/aptproxy/packages/ftp.de.debian.org_debian_dists_testing_InRelease > > file size mismatch (found 6631, expected 250532). Renaming to > > /var/cache/aptproxy/packages/ftp.de.debian.org_debian_dists_testing_InRelease.corrupted. > > I am sorry, but I can't reproduce this at all on stable or testing. > > Let's go back to basics. Could you enable debugging: > > debug = 1 > > restart the daemon and then on the apt-cacher box use wget to retrieve a > single file: > > wget http://localhost:/ftp.de.debian.org/debian/dists/testing/InRelease Ok, this started promissing, but then after 1% the time estimated increases quickly. Then suddenly it was 7%. Then (after ~ 10 Minutes) I aborted it. > If that fails, post the contents of /var/log/apt-cacher/error.log relating to > that request. I attached it to this email. On Sat, Aug 27, 2016 at 10:06:48AM +0100, Mark Hindley wrote: > > I am sorry, but I can't reproduce this at all on stable or testing. > > Also, as you are using port , does that mean you have tried approx as > well? No. Aeons ago I used apt-proxy and then switched to apt-cacher-ng. I left the port on all machines and configured apt-cacher-ng appropriately. However, apt-cacher-ng failes to install on this machine (#772489) and since I'm needing a proxy I thought I try apt-cacher next. > If so, can you check the ownership and permissions on /var/cache/aptproxy. drwxr-xr-x 8 www-data www-data 4,0K Aug 14 14:20 /var/cache/aptproxy Greetings Helge -- Dr. Helge Kreutzmann deb...@helgefjell.de Dipl.-Phys. http://www.helgefjell.de/debian.php 64bit GNU powered gpg signed mail preferred Help keep free software "libre": http://www.ffii.de/ Sun Aug 28 16:22:13 2016|warn [2377]: Info: /var/run/apt-cacher/ missing. Doing mkdir -p /var/run/apt-cacher/ Sun Aug 28 16:22:13 2016|info [2377]: Forked listener 2709 Sun Aug 28 16:34:39 2016|debug [6468]: Using IPv6 Sun Aug 28 16:34:39 2016|debug [6468]: Listening on ::: Sun Aug 28 16:34:39 2016|debug [6468]: fork listener Sun Aug 28 16:34:39 2016|info [6468]: Forked listener 6469 Sun Aug 28 16:34:39 2016|debug [6468]: Clean up before exiting. Sun Aug 28 16:34:39 2016|debug [6469]: Apt-Cacher version 1.7.13 started with Debug output enabled Sun Aug 28 16:34:52 2016|debug [6469]: Connection from :::127.0.0.1 Sun Aug 28 16:34:52 2016|debug [6472]: New Daemon connection Sun Aug 28 16:34:52 2016|debug [6472]: client is IPv4 mapped IPv6 address: mapping IPv4 configuration items to IPv6 Sun Aug 28 16:34:52 2016|debug [6472]: Test client 0:0:0:0:0::7F00:1/128 against localhost: 0:0:0:0:0::7F00:1/104 Sun Aug 28 16:34:52 2016|debug [6472]: Client is localhost Sun Aug 28 16:34:52 2016|debug [6472]: Processing a new request line Sun Aug 28 16:34:52 2016|debug [6472]: got: 'GET /ftp.de.debian.org/debian/dists/testing/InRelease HTTP/1.1' Sun Aug 28 16:34:52 2016|debug [6472]: Processing a new request line Sun Aug 28 16:34:52 2016|debug [6472]: got: 'User-Agent: Wget/1.18 (linux-gnu)' Sun Aug 28 16:34:52 2016|debug [6472]: Processing a new request line Sun Aug 28 16:34:52 2016|debug [6472]: got: 'Accept: */*' Sun Aug 28 16:34:52 2016|debug [6472]: Processing a new request line Sun Aug 28 16:34:52 2016|debug [6472]: got: 'Accept-Encoding: identity' Sun Aug 28 16:34:52 2016|debug [6472]: Processing a new request line Sun Aug 28 16:34:52 2016|debug [6472]: got: 'Host: localhost:' Sun Aug 28 16:34:52 2016|debug [6472]: Processing a new request line Sun Aug 28 16:34:52 2016|debug [6472]: got: 'Connection: Keep-Alive' Sun Aug 28 16:34:52 2016|debug [6472]: Processing a new request line Sun Aug 28 16:34:52 2016|debug [6472]: got: '' Sun Aug 28 16:34:52 2016|debug [6472]: Resolved request is http://ftp.de.debian.org/debian/dists/testing/InRelease Sun Aug 28 16:34:52 2016|debug [6472]: Index file: ftp.de.debian.org_debian_dists_testing_InRelease Sun Aug 28 16:34:52 2016|debug [6472]: Locked header
Bug#834968: apt-cacher: Fails to run due to curl errors
> I am sorry, but I can't reproduce this at all on stable or testing. Also, as you are using port , does that mean you have tried approx as well? If so, can you check the ownership and permissions on /var/cache/aptproxy. Thanks. Mark
Bug#834968: apt-cacher: Fails to run due to curl errors
On Thu, Aug 25, 2016 at 09:19:14PM +0200, Helge Kreutzmann wrote: > I try this, but this gives an endless loop of lines: > Holen:1 http://172.16.18.51:/ftp.de.debian.org/debian testing InRelease > [251 kB] > > (after some minutes, the next same line comes up, I left it running > for hours without any visual change). > > In the error.log I see: > Thu Aug 25 21:14:23 2016|info [11546]: return_file > /var/cache/aptproxy/packages/ftp.de.debian.org_debian_dists_testing_InRelease > aborted by timeout at 6631 of 250532 bytes > Thu Aug 25 21:14:27 2016|info [11547]: Warning: libcurl failed for > http://ftp.de.debian.org/debian/dists/testing/InRelease with status Timeout > was reached > Thu Aug 25 21:14:28 2016|info [11573]: ALARM! > /var/cache/aptproxy/packages/ftp.de.debian.org_debian_dists_testing_InRelease > file size mismatch (found 6631, expected 250532). Renaming to > /var/cache/aptproxy/packages/ftp.de.debian.org_debian_dists_testing_InRelease.corrupted. I am sorry, but I can't reproduce this at all on stable or testing. Let's go back to basics. Could you enable debugging: debug = 1 restart the daemon and then on the apt-cacher box use wget to retrieve a single file: wget http://localhost:/ftp.de.debian.org/debian/dists/testing/InRelease If that fails, post the contents of /var/log/apt-cacher/error.log relating to that request. Thanks. Mark
Bug#834968: apt-cacher: Fails to run due to curl errors
Hello Mark, On Tue, Aug 23, 2016 at 08:41:01AM +0100, Mark Hindley wrote: > > Looking at /var/log/apt-cacher I see lots of lines like Sun Aug 21 06:56:28 > > 2016|info [7482]: Warning: libcurl failed for > > http://debian/dists/testing/main/i18n/Translation-de with +status Peer > > certificate cannot be authenticated with given CA certificates > > OK, I think I have misunderstood this and the CA error has made me miss the > obvious -- Sorry! > > On Mon, Aug 22, 2016 at 07:43:59PM +0200, Helge Kreutzmann wrote: > > Hello Mark, > > On Mon, Aug 22, 2016 at 11:20:06AM +0100, Mark Hindley wrote: > > > Thanks. I am wondering where your HTTPS upstream is configured? What do > > > you have in /etc/apt/sources.list? > > > > root@samd:~# grep -v "^#" /etc/apt/sources.list > > deb http://172.16.18.51:/debian testing main > > You need to configure your upstream host somewhere. Probably the easiest for > you > is to add the host to sources.list: > >deb http://172.16.18.51:/upstream.host.fdqn/debian testing main > > This is documented as accessing apt-cacher as a mirror in the manpage. I try this, but this gives an endless loop of lines: Holen:1 http://172.16.18.51:/ftp.de.debian.org/debian testing InRelease [251 kB] (after some minutes, the next same line comes up, I left it running for hours without any visual change). In the error.log I see: Thu Aug 25 21:14:23 2016|info [11546]: return_file /var/cache/aptproxy/packages/ftp.de.debian.org_debian_dists_testing_InRelease aborted by timeout at 6631 of 250532 bytes Thu Aug 25 21:14:27 2016|info [11547]: Warning: libcurl failed for http://ftp.de.debian.org/debian/dists/testing/InRelease with status Timeout was reached Thu Aug 25 21:14:28 2016|info [11573]: ALARM! /var/cache/aptproxy/packages/ftp.de.debian.org_debian_dists_testing_InRelease file size mismatch (found 6631, expected 250532). Renaming to /var/cache/aptproxy/packages/ftp.de.debian.org_debian_dists_testing_InRelease.corrupted. Somehow still a libcurl problem? > Alternatively you could create a path_map setting: > >path_map = debian upstream.host.fqdn/debian Did not try. > Do you have a local DNS that means that debian resolves as a hostname? No, I only have /etc/hosts (this is a recent install, so no "cruft" lying around). Switching to direct net access still works flawlessly. Thanks! Greetings Helge -- Dr. Helge Kreutzmann deb...@helgefjell.de Dipl.-Phys. http://www.helgefjell.de/debian.php 64bit GNU powered gpg signed mail preferred Help keep free software "libre": http://www.ffii.de/ signature.asc Description: Digital signature
Bug#834968: apt-cacher: Fails to run due to curl errors
> Looking at /var/log/apt-cacher I see lots of lines like Sun Aug 21 06:56:28 > 2016|info [7482]: Warning: libcurl failed for > http://debian/dists/testing/main/i18n/Translation-de with +status Peer > certificate cannot be authenticated with given CA certificates OK, I think I have misunderstood this and the CA error has made me miss the obvious -- Sorry! On Mon, Aug 22, 2016 at 07:43:59PM +0200, Helge Kreutzmann wrote: > Hello Mark, > On Mon, Aug 22, 2016 at 11:20:06AM +0100, Mark Hindley wrote: > > Thanks. I am wondering where your HTTPS upstream is configured? What do you > > have in /etc/apt/sources.list? > > root@samd:~# grep -v "^#" /etc/apt/sources.list > deb http://172.16.18.51:/debian testing main You need to configure your upstream host somewhere. Probably the easiest for you is to add the host to sources.list: deb http://172.16.18.51:/upstream.host.fdqn/debian testing main This is documented as accessing apt-cacher as a mirror in the manpage. Alternatively you could create a path_map setting: path_map = debian upstream.host.fqdn/debian Do you have a local DNS that means that debian resolves as a hostname? Best wishes Mark
Bug#834968: apt-cacher: Fails to run due to curl errors
Hello Mark, On Mon, Aug 22, 2016 at 11:20:06AM +0100, Mark Hindley wrote: > Thanks. I am wondering where your HTTPS upstream is configured? What do you > have in /etc/apt/sources.list? root@samd:~# grep -v "^#" /etc/apt/sources.list deb http://172.16.18.51:/debian testing main Where 172.16.18.51 is the local IP of this machine. > On Sun, Aug 21, 2016 at 05:03:29PM +0200, Helge Kreutzmann wrote: > > > Are you trying to use HTTPS for upstream? If so, you will need to set > > > curl_ssl_insecure or use the libcurl option to configure CA to verify the > > > upstream connection. > > > > Is there some documentation to do this libcurl configuration? I'd > > rather configure the CA properly, but could not find any docs for > > this. > > Temporarily try the curl_ssl_insecure option first. If that helps we know we > are on the right track. Ok, I added: libcurl = insecure or libcurl = curl_ssl_insecure to /etc/apt-cacher/apt-cacher.conf but, unfortunately this does not change anything. > The libcurl option in mentioned in man apt-cacher(8). It should accept > anything from curl_easy_setopt(3), which requires the libcurl dev package I installed libcurl4-doc but the curl_easy_setopt(3) does not specify anything with INSECURE or insecure, nor did I find anything amongst the many options under the heading of SSL and SECURITY OPTIONS Could you check? Thanks! Greetings Helge -- Dr. Helge Kreutzmann deb...@helgefjell.de Dipl.-Phys. http://www.helgefjell.de/debian.php 64bit GNU powered gpg signed mail preferred Help keep free software "libre": http://www.ffii.de/ signature.asc Description: Digital signature
Bug#834968: apt-cacher: Fails to run due to curl errors
Thanks. I am wondering where your HTTPS upstream is configured? What do you have in /etc/apt/sources.list? On Sun, Aug 21, 2016 at 05:03:29PM +0200, Helge Kreutzmann wrote: > > Are you trying to use HTTPS for upstream? If so, you will need to set > > curl_ssl_insecure or use the libcurl option to configure CA to verify the > > upstream connection. > > Is there some documentation to do this libcurl configuration? I'd > rather configure the CA properly, but could not find any docs for > this. Temporarily try the curl_ssl_insecure option first. If that helps we know we are on the right track. The libcurl option in mentioned in man apt-cacher(8). It should accept anything from curl_easy_setopt(3), which requires the libcurl dev package Mark
Bug#834968: apt-cacher: Fails to run due to curl errors
Hello Mark, On Sun, Aug 21, 2016 at 10:58:50AM +0100, Mark Hindley wrote: > On Sun Aug 21 06:04:12 2016 Helge Kreutzmannwrote: > > Package: apt-cacher > > Version: 1.7.13 > > Severity: normal > > > > I wanted to use apt-cacher-ng. > > This is apt-cacher not apt-cacher-ng! Yes, sorry. Mixed packages. > > However, when trying to "apt-get update" I get the following error: > > root@samd:/var/log/apt-cacher# LC_ALL=C apt-get update > > Ign:1 http://172.16.18.51:/debian testing InRelease > > This looks like you also have a path_map setting? It isn't included in the > /etc/apt-cacher/apt-cacher.conf in the bug report. Do you have any > configuration fragments in /etc/apt-cacher/conf.d? apt-cacher.conf is essentially: root@samd:~# cat /etc/apt-cacher/apt-cacher.conf | grep -v "^#"|grep -v "^$" cache_dir = /var/cache/aptproxy admin_email = helge@localhost daemon_port = group = www-data user = www-data checksum = 1 allowed_hosts = 172.16.18.0/8 /etc/apt-cacher/conf.d/ is empty. root@samd:~# grep -r path_map /etc/apt-cacher/* /etc/apt-cacher/apt-cacher.conf:#path_map = debian ftp.uni-kl.de/pub/linux/debian ftp2.de.debian.org/debian ; /etc/apt-cacher/apt-cacher.conf:# There are 2 default internal path_map settings for the Debian and Ubuntu /etc/apt-cacher/apt-cacher.conf:#path_map = debian-changelogs /etc/apt-cacher/apt-cacher.conf:# directories whose names are derived from the relevant path_map key. So /etc/apt-cacher/apt-cacher.conf:# generally there will be a path_map key => server(s) setting for each /etc/apt-cacher/apt-cacher.conf:# the URLs. This is matched before any path_map settings are expanded. If /etc/apt-cacher/apt-cacher.conf:# the path_map setting. Note these items are strings, not regexps. > Are you trying to use HTTPS for upstream? If so, you will need to set > curl_ssl_insecure or use the libcurl option to configure CA to verify the > upstream connection. Is there some documentation to do this libcurl configuration? I'd rather configure the CA properly, but could not find any docs for this. Thanks also for you speedy reply. Greetings Helge -- Dr. Helge Kreutzmann deb...@helgefjell.de Dipl.-Phys. http://www.helgefjell.de/debian.php 64bit GNU powered gpg signed mail preferred Help keep free software "libre": http://www.ffii.de/ signature.asc Description: Digital signature
Bug#834968: apt-cacher: Fails to run due to curl errors
Helge, Thanks. On Sun Aug 21 06:04:12 2016 Helge Kreutzmannwrote: > Package: apt-cacher > Version: 1.7.13 > Severity: normal > > I wanted to use apt-cacher-ng. This is apt-cacher not apt-cacher-ng! > However, when trying to "apt-get update" I get the following error: > root@samd:/var/log/apt-cacher# LC_ALL=C apt-get update > Ign:1 http://172.16.18.51:/debian testing InRelease This looks like you also have a path_map setting? It isn't included in the /etc/apt-cacher/apt-cacher.conf in the bug report. Do you have any configuration fragments in /etc/apt-cacher/conf.d? Are you trying to use HTTPS for upstream? If so, you will need to set curl_ssl_insecure or use the libcurl option to configure CA to verify the upstream connection. Best wishes Mark
Bug#834968: apt-cacher: Fails to run due to curl errors
Package: apt-cacher Version: 1.7.13 Severity: normal I wanted to use apt-cacher-ng. I installed it and set up the data as I assumed appropriate. However, when trying to "apt-get update" I get the following error: root@samd:/var/log/apt-cacher# LC_ALL=C apt-get update Ign:1 http://172.16.18.51:/debian testing InRelease Ign:2 http://172.16.18.51:/debian testing Release Ign:3 http://172.16.18.51:/debian testing/main amd64 Packages Ign:4 http://172.16.18.51:/debian testing/main all Packages Ign:5 http://172.16.18.51:/debian testing/main Translation-en Ign:6 http://172.16.18.51:/debian testing/main Translation-de_DE Ign:7 http://172.16.18.51:/debian testing/main Translation-de Ign:8 http://172.16.18.51:/debian testing/main amd64 DEP-11 Metadata Ign:9 http://172.16.18.51:/debian testing/main all DEP-11 Metadata Ign:10 http://172.16.18.51:/debian testing/main DEP-11 64x64 Icons Ign:11 http://172.16.18.51:/debian testing/main amd64 Contents (deb) Ign:12 http://172.16.18.51:/debian testing/main all Contents (deb) Ign:3 http://172.16.18.51:/debian testing/main amd64 Packages Ign:4 http://172.16.18.51:/debian testing/main all Packages Ign:5 http://172.16.18.51:/debian testing/main Translation-en Ign:6 http://172.16.18.51:/debian testing/main Translation-de_DE Ign:7 http://172.16.18.51:/debian testing/main Translation-de Ign:8 http://172.16.18.51:/debian testing/main amd64 DEP-11 Metadata Ign:9 http://172.16.18.51:/debian testing/main all DEP-11 Metadata Ign:10 http://172.16.18.51:/debian testing/main DEP-11 64x64 Icons Ign:11 http://172.16.18.51:/debian testing/main amd64 Contents (deb) Ign:12 http://172.16.18.51:/debian testing/main all Contents (deb) Ign:3 http://172.16.18.51:/debian testing/main amd64 Packages Ign:4 http://172.16.18.51:/debian testing/main all Packages Ign:5 http://172.16.18.51:/debian testing/main Translation-en Ign:6 http://172.16.18.51:/debian testing/main Translation-de_DE Ign:7 http://172.16.18.51:/debian testing/main Translation-de Ign:8 http://172.16.18.51:/debian testing/main amd64 DEP-11 Metadata Ign:9 http://172.16.18.51:/debian testing/main all DEP-11 Metadata Ign:10 http://172.16.18.51:/debian testing/main DEP-11 64x64 Icons Ign:11 http://172.16.18.51:/debian testing/main amd64 Contents (deb) Ign:12 http://172.16.18.51:/debian testing/main all Contents (deb) Ign:3 http://172.16.18.51:/debian testing/main amd64 Packages Ign:4 http://172.16.18.51:/debian testing/main all Packages Ign:5 http://172.16.18.51:/debian testing/main Translation-en Ign:6 http://172.16.18.51:/debian testing/main Translation-de_DE Ign:7 http://172.16.18.51:/debian testing/main Translation-de Ign:8 http://172.16.18.51:/debian testing/main amd64 DEP-11 Metadata Ign:9 http://172.16.18.51:/debian testing/main all DEP-11 Metadata Ign:10 http://172.16.18.51:/debian testing/main DEP-11 64x64 Icons Ign:11 http://172.16.18.51:/debian testing/main amd64 Contents (deb) Ign:12 http://172.16.18.51:/debian testing/main all Contents (deb) Ign:3 http://172.16.18.51:/debian testing/main amd64 Packages Ign:4 http://172.16.18.51:/debian testing/main all Packages Ign:5 http://172.16.18.51:/debian testing/main Translation-en Ign:6 http://172.16.18.51:/debian testing/main Translation-de_DE Ign:7 http://172.16.18.51:/debian testing/main Translation-de Ign:8 http://172.16.18.51:/debian testing/main amd64 DEP-11 Metadata Ign:9 http://172.16.18.51:/debian testing/main all DEP-11 Metadata Ign:10 http://172.16.18.51:/debian testing/main DEP-11 64x64 Icons Ign:11 http://172.16.18.51:/debian testing/main amd64 Contents (deb) Ign:12 http://172.16.18.51:/debian testing/main all Contents (deb) Err:3 http://172.16.18.51:/debian testing/main amd64 Packages 502 apt-cacher: libcurl error: Peer certificate cannot be authenticated with given CA certificates Ign:4 http://172.16.18.51:/debian testing/main all Packages Ign:5 http://172.16.18.51:/debian testing/main Translation-en Ign:6 http://172.16.18.51:/debian testing/main Translation-de_DE Ign:7 http://172.16.18.51:/debian testing/main Translation-de Ign:8 http://172.16.18.51:/debian testing/main amd64 DEP-11 Metadata Ign:9 http://172.16.18.51:/debian testing/main all DEP-11 Metadata Ign:10 http://172.16.18.51:/debian testing/main DEP-11 64x64 Icons Ign:11 http://172.16.18.51:/debian testing/main amd64 Contents (deb) Ign:12 http://172.16.18.51:/debian testing/main all Contents (deb) Reading package lists... Done W: The repository 'http://172.16.18.51:/debian testing Release' does not have a Release file. N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use. N: See apt-secure(8) manpage for repository creation and user configuration details. E: Failed to fetch