Bug#501747: apt-cacher: clients take excessive time to download files
I think I have tracked this down. Could you try this patch for
/usr/share/apt-cacher/apt-cacher (against 1.6.6.) and let me know.
Mark
diff --git a/apt-cacher b/apt-cacher
index 1fb7ab7..765b240 100755
--- a/apt-cacher
+++ b/apt-cacher
@@ -1211,7 +1211,7 @@ sub connect_curlm {
}
}
# Check for pending new request
- if ($active_handles $select-can_read(0.1)) {
+ if ($active_handles $select-can_read(0)) {
debug_message('Pending connection');
next LIBCURL_REQUEST;
}
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501747: apt-cacher: clients take excessive time to download files
Package: apt-cacher tag 501747 pending thanks On Sat, Oct 18, 2008 at 02:27:11PM +0200, Guntsche Michael wrote: On Oct 18, 2008, at 12:27, Mark Hindley wrote: I think I have tracked this down. Could you try this patch for /usr/share/apt-cacher/apt-cacher (against 1.6.6.) and let me know. Hello Mark, That little change did it I am now back to a normal download rate ~600K. thank you very much for looking into this. /Michael Great. I will queue that for the next upload Mark -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501747: apt-cacher: clients take excessive time to download files
On Wed, 15 Oct 2008 22:58:04 +0100, Mark Hindley [EMAIL PROTECTED] wrote: I am afraid I can't reproduce this behaviour (admittedly I have the apt-cacher version 1.6.6 which is still to be uploaded. If you want to try it and see if it is any better you can get it from http://www.hindley.org.uk/~mark/debian). I assume you have found this with 1.6.4? The site is down at the moment but I will try it as soon as I can fetch it You can use wget/curl to download files directly through the cache: http://localhost:3142/debian/pool/main/t/tcpdump/tcpdump_3.9.8-4_i386.deb Thanks for the information, I tried this with the gimp package against ftp.at.kernel.org. Running via the proxy gives me 147KB, fetching it directly is ~500-600KB. So there is really something going on with the proxy here. As soon as I am able to fetch the new version I will run a debug run with it. /Michael -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501747: apt-cacher: clients take excessive time to download files
On Thu, Oct 16, 2008 at 11:26:03AM +0200, Michael Guntsche wrote: I do not see anything fishy here but i KNOW that a while back I had the full download speed (back then around 330K). So I do not understand what's happening here. Thanks. It basically looks fine, appart from the frequent read 0 bytes, which suggests that return_file() is often idle. So the bottleneck is before there. Do you know which version gave you decent speed? It is difficult for me to test, as the limit is above the speed of my ADSL connection, so I can't reproduce it! Will give it some more thought. Mark -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501747: apt-cacher: clients take excessive time to download files
Could you apply this patch to /usr/share/apt-cacher/apt-cacher and see
if it makes any difference?
Mark
diff --git a/apt-cacher b/apt-cacher
index 1fb7ab7..9d2e7b2 100755
--- a/apt-cacher
+++ b/apt-cacher
@@ -1332,7 +1332,7 @@ sub libcurl {
}
elsif ($pkfdref) {
my $pkfd=$$pkfdref;
- print $pkfd $_;
+ syswrite $pkfd, $_;
data_feed(\$_);
}
}
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501747: apt-cacher: clients take excessive time to download files
On Thu, Oct 16, 2008 at 01:19:40PM +0200, Michael Guntsche wrote: On Thu, 16 Oct 2008 12:04:07 +0100, Mark Hindley [EMAIL PROTECTED] wrote: I had thought your problem was with 1.6.4. The OP of ths bug identifies 1.6.4 as problematic. Is it fine for you? 1.6.4 is fine for me. Sorry for not reading the bugreport in more detail. I just saw the topic and read through his problems but never noticed that he was actually talking about 1.6.4. Nevertheless I tried it once again 1.6.4 is really fast and I do not experience any problems. Can it be the change of having curl run in his own process (just read through the Changelog). Yes, I suspect that too. I think it may be a related to buffered/non-buffered IO. That is why the little patch I sent you helped a bit. I will do some more work. Mark -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501747: apt-cacher: clients take excessive time to download files
On Thu, Oct 16, 2008 at 12:53:32PM +0200, Michael Guntsche wrote: On Thu, 16 Oct 2008 11:28:54 +0100, Mark Hindley [EMAIL PROTECTED] wrote: Do you know which version gave you decent speed? Hah, that was easier than I expected. 1.6.4 gives me decent performance, I am not up to my full bandwith but loading along with 500K So it seems this is an internal problem with apt-cacher. I had thought your problem was with 1.6.4. The OP of ths bug identifies 1.6.4 as problematic. Is it fine for you? see http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=501747#38 Mark -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501747: apt-cacher: clients take excessive time to download files
Hi, I seem to have a similar problem. In my case apt-get update works rather fast. If I then download a package via apt-get install I get a download rate of 167K if I got directly to the same server (removing the proxy entries in sources.list) I get around 560K of download. This started recently so I it might be the upgrade of another package that cause this problem. is there a way to just test the package download part of apt-cacher? Kind regards, michael -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501747: apt-cacher: clients take excessive time to download files
On Wed, Oct 15, 2008 at 09:22:11PM +0200, Guntsche Michael wrote: Hi, I seem to have a similar problem. In my case apt-get update works rather fast. If I then download a package via apt-get install I get a download rate of 167K if I got directly to the same server (removing the proxy entries in sources.list) I get around 560K of download. This started recently so I it might be the upgrade of another package that cause this problem. is there a way to just test the package download part of apt-cacher? Thanks. I am afraid I can't reproduce this behaviour (admittedly I have the apt-cacher version 1.6.6 which is still to be uploaded. If you want to try it and see if it is any better you can get it from http://www.hindley.org.uk/~mark/debian). I assume you have found this with 1.6.4? You can use wget/curl to download files directly through the cache: --22:46:30-- http://localhost:3142/debian/pool/main/t/tcpdump/tcpdump_3.9.8-4_i386.deb = `/dev/null' Resolving localhost... 127.0.0.1 Connecting to localhost|127.0.0.1|:3142... connected. HTTP request sent, awaiting response... 200 (OK) OK Length: 305,698 (299K) [application/x-debian-package] 0K .. .. .. .. .. 16% 139.37 MB/s 50K .. .. .. .. .. 33% 49.89 KB/s 100K .. .. .. .. .. 50% 202.52 MB/s 150K .. .. .. .. .. 66% 49.89 KB/s 200K .. .. .. .. .. 83% 288.41 MB/s 250K .. .. .. .. 100% 48.26 KB/s 22:46:35 (99.16 KB/s) - `/dev/null' saved [305698/305698] --22:47:21-- http://ftp.uk.debian.org/debian/pool/main/t/tcpdump/tcpdump_3.9.8-4_i386.deb = `/dev/null' Resolving ftp.uk.debian.org... 83.142.228.128 Connecting to ftp.uk.debian.org|83.142.228.128|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 305,698 (299K) [application/x-debian-package] 0K .. .. .. .. .. 16% 79.95 KB/s 50K .. .. .. .. .. 33% 85.33 KB/s 100K .. .. .. .. .. 50% 84.50 KB/s 150K .. .. .. .. .. 66% 109.17 KB/s 200K .. .. .. .. .. 83% 106.08 KB/s 250K .. .. .. .. 100% 118.30 KB/s 22:47:25 (94.99 KB/s) - `/dev/null' saved [305698/305698] The limit for my ADSL is about 125KB/s, so I am not that far off. Maybe it is only seen at higher speeds. I really need some debug info when this behaviours is seen. Could you set debug=1 in the config file. Restart the daemon and then fetch a single file through the cache. Send me the info in /var/log/apt-cacher/error.log Thanks, Mark -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501747: apt-cacher: clients take excessive time to download files
On Mon, Oct 13, 2008 at 09:53:25AM +1100, Ben Finney wrote: package apt-cacher found 501747 1.6.4 thanks On 12-Oct-2008, Mark Hindley wrote: Can you provide me with your apt-cacher version = $ aptitude show apt-cacher | grep '^Version:' Version: 1.6.4 = which mode are you using (daemon, inetd or cgi) I don't know exactly which configuration setting determines that, so here are all the configuration settings for ???apt-cacher???: Thanks. What does /etc/default/apt-cacher contain? How have you pointed apt to the cache? Changed /etc/apt/sources.list or set Acquire::htp::proxy in /etc/apt/apt.conf? and the debug log output when you are doing the apt-get update through the cache that is so slow. The debug log doesn't currently exist (at least, I see no file named ???/var/log/apt-cacher/debug.log???). What specifically do I need to do to get that? Set debug=0 in /etc/apt-cacher/apt-cacher.conf Thanks Mark -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501747: apt-cacher: clients take excessive time to download files
On 13-Oct-2008, Mark Hindley wrote: On Mon, Oct 13, 2008 at 09:53:25AM +1100, Ben Finney wrote: I don't know exactly which configuration setting determines that, so here are all the configuration settings for ???apt-cacher???: Thanks. What does /etc/default/apt-cacher contain? = $ grep -v '^\(#.*\|\s*\)$' /etc/default/apt-cacher AUTOSTART=1 = How have you pointed apt to the cache? Changed /etc/apt/sources.list or set Acquire::htp::proxy in /etc/apt/apt.conf? I wasn't aware that ‘apt-cacher’ works as an HTTP proxy, I'd love to know how to use it that way! I thought one needs to modify the requested URLs in ‘sources.list’ directly. So, here is the configuration (currently disabled) on the machine that has slow performance via ‘apt-cacher’: = $ grep -v '^\(#.*\|\s*\)$' /etc/apt/sources.list.d/whitetree-lan.list.DISABLED deb http://proxy:3142/security.debian.org/debian-security/ lenny/updates main deb http://proxy:3142/mirror.internode.on.net/pub/debian/ lenny main = The above configuration (when enabled by renaming the file) causes the behaviour reported in this bug report; so, I've currently got that machine configured with this: = $ grep -v '^\(#.*\|\s*\)$' /etc/apt/sources.list.d/whitetree-lan.NOPROXY.list deb http://security.debian.org/debian-security/ lenny/updates main deb http://mirror.internode.on.net/pub/debian/ lenny main = The above configuration works properly, in contrast to the configuration using the proxy. The debug log doesn't currently exist (at least, I see no file named ???/var/log/apt-cacher/debug.log???). What specifically do I need to do to get that? Set debug=0 in /etc/apt-cacher/apt-cacher.conf Its configuration already has “debug=0”. I presume you mean something other than that value? Perhaps “debug=1” or something else? Once I've changed the configuration, is there some particular sequence of actions you want me to perform before giving the resulting debug log output? -- \ “Only the educated are free.” —Epictetus, _Discourses_ | `\ | _o__) | Ben Finney [EMAIL PROTECTED] signature.asc Description: Digital signature
Bug#501747: apt-cacher: clients take excessive time to download files
On Mon, Oct 13, 2008 at 06:46:07PM +1100, Ben Finney wrote: I wasn't aware that ???apt-cacher??? works as an HTTP proxy, I'd love to know how to use it that way! I thought one needs to modify the requested URLs in ???sources.list??? directly. Yes, you could leave sources.list unchanged and set Acquire::http::Proxy in apt.conf. See man apt-cacher(1) and apt.conf(5) So, here is the configuration (currently disabled) on the machine that has slow performance via ???apt-cacher???: = $ grep -v '^\(#.*\|\s*\)$' /etc/apt/sources.list.d/whitetree-lan.list.DISABLED deb http://proxy:3142/security.debian.org/debian-security/ lenny/updates main deb http://proxy:3142/mirror.internode.on.net/pub/debian/ lenny main = The above configuration (when enabled by renaming the file) causes the behaviour reported in this bug report; so, I've currently got that machine configured with this: = $ grep -v '^\(#.*\|\s*\)$' /etc/apt/sources.list.d/whitetree-lan.NOPROXY.list deb http://security.debian.org/debian-security/ lenny/updates main deb http://mirror.internode.on.net/pub/debian/ lenny main = The above configuration works properly, in contrast to the configuration using the proxy. The debug log doesn't currently exist (at least, I see no file named ???/var/log/apt-cacher/debug.log???). What specifically do I need to do to get that? Set debug=0 in /etc/apt-cacher/apt-cacher.confo Its configuration already has ???debug=0???. I presume you mean something other than that value? Perhaps ???debug=1??? or something else? Sorry, that should be debug=1 :) Once I've changed the configuration, is there some particular sequence of actions you want me to perform before giving the resulting debug log output? Restore the disabled sources.list that triggers the bug and then do apt-get update. /var/log/apt-cacher/error.log should then contain debug info about what apt-cacher is doing. Mark -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501747: apt-cacher: clients take excessive time to download files
package apt-cacher found 501747 1.6.4 thanks On 12-Oct-2008, Mark Hindley wrote: Can you provide me with your apt-cacher version = $ aptitude show apt-cacher | grep '^Version:' Version: 1.6.4 = which mode are you using (daemon, inetd or cgi) I don't know exactly which configuration setting determines that, so here are all the configuration settings for ‘apt-cacher’: = $ grep -v '^\(#.*\|\s*\)$' /etc/apt-cacher/apt-cacher.conf cache_dir=/var/cache/apt-cacher [EMAIL PROTECTED] daemon_port=3142 group=www-data user=www-data allowed_hosts=192.168.5.0/24,192.168.7.0/24 denied_hosts= allowed_hosts_6=fec0::/16 denied_hosts_6= generate_reports=1 clean_cache=1 offline_mode=0 logdir=/var/log/apt-cacher expire_hours=0 http_proxy= use_proxy=0 http_proxy_auth= use_proxy_auth=0 limit=0 debug=0 = and the debug log output when you are doing the apt-get update through the cache that is so slow. The debug log doesn't currently exist (at least, I see no file named ‘/var/log/apt-cacher/debug.log’). What specifically do I need to do to get that? -- \ “Pinky, are you pondering what I'm pondering?” “I think so, | `\ Brain, but if it was only supposed to be a three hour tour, why | _o__) did the Howells bring all their money?” —_Pinky and The Brain_ | Ben Finney [EMAIL PROTECTED] signature.asc Description: Digital signature
