On Thu, Aug 18, 2016 at 04:47:55AM -0700, poiuytrez wrote:
> I have frequent errors when I do apt-get update: 
> W: Failed to fetch 
> http://http.debian.net/debian/dists/jessie-backports/main/source/Sources  
> Hash Sum mismatch

For you, I would suggest trying a different mirror.

$ host http.debian.net
http.debian.net is an alias for httpredir.debian.org.

This redirection service works very well for some people, and very poorly
for other people.  You appear to be in the latter group.

You can either select a single mirror directly from:
https://www.debian.org/mirror/list

(the single mirrors are listed at the bottom of the page)

or you can use one of the country-code round robin services, such as:
http://ftp.us.debian.org/debian/
http://ftp.ca.debian.org/debian/

(the round robins are listed at the top of the page).

Obviously you would want to choose a country that's close to you,
network-wise.

> Sometimes it works, sometimes it doesn't. The problem appeared on all my
> servers running Debian 8 (hosted on Google Cloud Platform) the last couple of
> days. 

I have also experienced a MASSIVE problem with apt-get in the last week
or so.  I don't know if it's the same problem you're having, or totally
unrelated, but I am going to piggyback on your thread.

I have my problem at WORK only.  At work, I am trapped inside a firewalled
network.  I have no control over the firewall.  The entities who control
it are mysterious dark wizards who do not reveal themselves or their
secrets to mere mortals like me.

One of the past features of this firewall was a "transparent" (worst term
ever) HTTP proxy with forced authentication (user name and password).
I was able to plead to get an exception in place.  HTTP connections coming
from my desktop workstation's IP address weren't required to perform a
Windows Active Directory login to the HTTP proxy.  So I put a Squid proxy
on my workstation and told all of the Debian servers to use that for apt.

This worked well enough for years.

Now, starting last week, the problem is that when I try to apt-get update,
the files coming from security.debian.org HANG at 100% complete.  It
looks something like this:

root@ebase-fla:~# apt-get update
Hit http://ftp.us.debian.org wheezy Release.gpg
Hit http://ftp.us.debian.org wheezy-updates Release.gpg                        
Hit http://ftp.us.debian.org wheezy Release                                    
Hit http://ftp.us.debian.org wheezy-updates Release                            
Hit http://ftp.us.debian.org wheezy/main Sources             
Hit http://ftp.us.debian.org wheezy/contrib Sources                            
Hit http://ftp.us.debian.org wheezy/non-free Sources                           
Hit http://ftp.us.debian.org wheezy/main amd64 Packages                        
Hit http://ftp.us.debian.org wheezy/contrib amd64 Packages                     
Hit http://ftp.us.debian.org wheezy/non-free amd64 Packages                    
Hit http://ftp.us.debian.org wheezy/contrib Translation-en                     
Hit http://ftp.us.debian.org wheezy/main Translation-en                        
Hit http://ftp.us.debian.org wheezy/non-free Translation-en                    
Hit http://ftp.us.debian.org wheezy-updates/main Sources                       
Hit http://ftp.us.debian.org wheezy-updates/contrib Sources                    
Hit http://ftp.us.debian.org wheezy-updates/non-free Sources                   
Hit http://ftp.us.debian.org wheezy-updates/main amd64 Packages/DiffIndex
Hit http://ftp.us.debian.org wheezy-updates/contrib amd64 Packages             
Hit http://ftp.us.debian.org wheezy-updates/non-free amd64 Packages/DiffIndex  
Hit http://ftp.us.debian.org wheezy-updates/contrib Translation-en             
Hit http://ftp.us.debian.org wheezy-updates/main Translation-en/DiffIndex      
Hit http://ftp.us.debian.org wheezy-updates/non-free Translation-en/DiffIndex  
Hit http://security.debian.org wheezy/updates Release.gpg                      
Hit http://security.debian.org wheezy/updates Release
Hit http://security.debian.org wheezy/updates/contrib Sources
Hit http://security.debian.org wheezy/updates/non-free Sources
Get:1 http://security.debian.org wheezy/updates/main amd64 Packages [426 kB]
Hit http://security.debian.org wheezy/updates/contrib amd64 Packages           
Hit http://security.debian.org wheezy/updates/non-free amd64 Packages
Hit http://security.debian.org wheezy/updates/contrib Translation-en
Hit http://security.debian.org wheezy/updates/non-free Translation-en          
Get:2 http://security.debian.org wheezy/updates/main Sources [276 kB]
Get:3 http://security.debian.org wheezy/updates/main amd64 Packages [426 kB]   
Ign http://security.debian.org wheezy/updates/main Translation-en              
Get:4 http://security.debian.org wheezy/updates/main Sources [276 kB]
Err http://security.debian.org wheezy/updates/main amd64 Packages
  404  Not Found [IP: 128.101.240.215 80]
Fetched 69.9 kB in 22min 8s (52 B/s)
W: Failed to fetch 
http://security.debian.org/dists/wheezy/updates/main/source/Sources  Hash Sum 
mismatch

W: Failed to fetch 
http://security.debian.org/dists/wheezy/updates/main/binary-amd64/Packages  404 
 Not Found [IP: 128.101.240.215 80]

E: Some index files failed to download. They have been ignored, or old ones 
used instead.


So, after 22 minutes of waiting for that to fail, I try again.  And again.
And again.  And again.  Every 20 to 30 minutes, which is how long it
takes to finally give up and die.

The "hanging at 100%" symptom isn't visible at the very end.  It's only
visible during the apt-get.  It gets erased from the terminal when apt
finally gives up.  That symptom looks like:

...
Get:3 http://security.debian.org wheezy/updates/main Sources [277 kB]
...
Hit http://ftp.us.debian.org wheezy-updates/main Translation-en/DiffIndex      
Hit http://ftp.us.debian.org wheezy-updates/non-free Translation-en/DiffIndex
100% [3 Sources 276 kB/277 kB 100%]_

where _ is the terminal's cursor.  Sitting.  Waiting.  Eternally.
(Or, for 20-30 minutes, which merely feels like eternity.)

I've tried bypassing the Squid proxy.  Maybe the dark wizards dropped the
Windows AD authentication requirement based on source IP, or destination
domain name, or the alignment of the planets, or god-only-knows-what.
It makes no difference.  I get the same symptoms whether I go through
my Squid proxy or not.

I've tried Googling, which led me to look into Acquire::http::No-Cache.

Here's one of my apt.conf attempts:

root@ebase-fla:~# cat /etc/apt/apt.conf
// Acquire::http::Proxy "http://imadev.eeg.ccf.org:3128/";;
Acquire::http::No-Cache "true";

This didn't change the behavior either.

Nothing I've tried has worked.  Occasionally after a few days of trying,
one of my machines will get LUCKY, but it is not reproducible.

Apparently there is also no way for me to use a SOCKS proxy (i.e. an
ssh -D tunnel) with apt, which is what I normally use for my web browsing.
Not that I would be wild about forcing all of the workplace servers to
use my private ssh -D tunnel for their legitimate business-relevant
security updates, but at least it would *work*.  Except that, y'know, it
doesn't actually work.  There isn't any apt-get-through-socks feature
that I can see.

To try to identify the dark wizards' proxy, I attempt a manual HTTP
session:

imadev:~$ telnet localhost 3128
Trying...
Connected to localhost.
Escape character is '^]'.
HEAD http://www.debian.org/ HTTP/1.0

HTTP/1.0 200 OK
Date: Thu, 18 Aug 2016 12:37:01 GMT
Server: Apache
Content-Location: index.en.html
Vary: negotiate,accept-language,Accept-Encoding
TCN: choice
Last-Modified: Thu, 18 Aug 2016 03:43:23 GMT
ETag: "39cf-53a5062360bac"
Accept-Ranges: bytes
Content-Length: 14799
Cache-Control: max-age=86400
Expires: Fri, 19 Aug 2016 12:37:01 GMT
X-Clacks-Overhead: GNU Terry Pratchett
Content-Type: text/html
Content-Language: en
Age: 0
X-Cache: MISS from imadev
Via: 1.1 wfmc-cluster, 1.0 imadev:3128 (squid/2.7.STABLE9)
Connection: close

Connection closed by foreign host.


That's all the information I have: "Via: 1.1 wfmc-cluster".  I've tried
Googling that, with no success.  Maybe "wfmc-cluster" is a hostname,
in which case I have zero information about what software it's running.
(No, it does not resolve.  I tried that.)

My Debian system at home, and my Debian VPS, do not have this issue.
Nobody on IRC seems to be having this issue.  My tentative conclusion
is that it must be related to my workplace's HTTP proxy.

If anyone has suggestions for how to get apt to work around a workplace's
"transparent" (as mud) HTTP proxy, I'd love to hear them.

Reply via email to