Bug#880645: wrong content in xz-compressed dep11 Components files (outdated)

2017-11-04 Thread Matthias Klumpp 
2017-11-03 11:26 GMT+01:00 Ansgar Burchardt :
> Julian Andres Klode writes:
> [...]
>>> E: Failed to fetch 
>>> store:/var/lib/apt/lists/partial/mirrors.wikimedia.org_debian_dists_unstable_non-free_dep11_Components-amd64.yml.xz
>>>   Hash Sum mismatch
>>>Hashes of expected file:
>>> - Filesize:65721 [weak]
>>> - 
>>> SHA256:0eb5ea70b4c46a7c0889e9601d16cedf3eb4ff55bf3b77ef2af2bcd3dcf48cec
>>> - MD5Sum:68fdfbc3bc806010ba4c9fc8d4cb133b [weak]
>>>Hashes of received file:
>>> - 
>>> SHA256:5fa520cd26eac94e183632f84371643219fdf905dfd351b53bbcd7afa5c3b848
>>> - MD5Sum:875dd6824d698bfa73c6517307f692f5 [weak]
>>> - Filesize:65721 [weak]
>>>Last modification reported: Mon, 30 Oct 2017 21:09:03 +
>>>Release file created at: Fri, 03 Nov 2017 02:41:19 +
>>> E: Some index files failed to download. They have been ignored, or old ones 
>>> used instead.
>>> ```
>>>
>> The archive is broken, .xz contains different content than .gz - writing the 
>> .xz file
>> failed. The are several .xz.new files, so I expect some of the other files 
>> are broken
>> too.
>
> The appstream generator crashed and left inconsistent output data
> (updates .gz, but old .xz) already on mekeel.d.o...  That is really not
> nice: the data synced into the archive should really be in a staging
> location and only be updated once the generator finished successfully.
>
> CC'ed mak@ as he maintains the appstream generator.

Hmm, I didn't know APT would have issues with this - I guess to be
safe it would make sense to rsync the data directory to a place where
dak can fetch it from after appstream-generator has finished
successfully.
Alternatively, I could maybe change the code so the contents of the
.gz/.xz files are ensured to be always identical.

In any case, there is bug#880645 to track this now.

Cheers,
Matthias


-- 
I welcome VSRE emails. See http://vsre.info/

Bug#880645: wrong content in xz-compressed dep11 Components files (outdated)

2017-11-03 Thread Ansgar Burchardt 
Julian Andres Klode writes:
[...]
>> E: Failed to fetch 
>> store:/var/lib/apt/lists/partial/mirrors.wikimedia.org_debian_dists_unstable_non-free_dep11_Components-amd64.yml.xz
>>   Hash Sum mismatch
>>Hashes of expected file:
>> - Filesize:65721 [weak]
>> - SHA256:0eb5ea70b4c46a7c0889e9601d16cedf3eb4ff55bf3b77ef2af2bcd3dcf48cec
>> - MD5Sum:68fdfbc3bc806010ba4c9fc8d4cb133b [weak]
>>Hashes of received file:
>> - SHA256:5fa520cd26eac94e183632f84371643219fdf905dfd351b53bbcd7afa5c3b848
>> - MD5Sum:875dd6824d698bfa73c6517307f692f5 [weak]
>> - Filesize:65721 [weak]
>>Last modification reported: Mon, 30 Oct 2017 21:09:03 +
>>Release file created at: Fri, 03 Nov 2017 02:41:19 +
>> E: Some index files failed to download. They have been ignored, or old ones 
>> used instead.
>> ```
>> 
> The archive is broken, .xz contains different content than .gz - writing the 
> .xz file
> failed. The are several .xz.new files, so I expect some of the other files 
> are broken
> too.

The appstream generator crashed and left inconsistent output data
(updates .gz, but old .xz) already on mekeel.d.o...  That is really not
nice: the data synced into the archive should really be in a staging
location and only be updated once the generator finished successfully.

CC'ed mak@ as he maintains the appstream generator.

Ansgar

Bug#880645: wrong content in xz-compressed dep11 Components files (outdated) (was: apt: Hash Sum mismatch when running apt-get update)

2017-11-03 Thread Julian Andres Klode 
Control: reassign -1 ftp.debian.org
Control: retitle -1 wrong content in xz-compressed dep11 Components files 
(outdated)

On Fri, Nov 03, 2017 at 10:54:49AM +0200, Ilan Cohen wrote:
> Package: apt
> Version: 1.6~alpha3
> Severity: important
> 
> Dear Maintainer,
> 
>* What led up to the situation?
> 
> Running `apt-get update` gave me the following output:
> 
> ```
> Hit:1 http://security.debian.org/debian-security buster/updates InRelease
> Hit:2 http://ftp.us.debian.org/debian buster InRelease
>   
>   
> Hit:3 https://download.docker.com/linux/debian stretch InRelease  
>   
> 
> Ign:4 https://dl.google.com/linux/chrome/deb stable InRelease 
>   
> Hit:5 http://ftp.us.debian.org/debian buster-updates InRelease
>   
> Hit:6 https://dl.google.com/linux/chrome/deb stable Release  
> Hit:7 https://mirrors.wikimedia.org/debian experimental InRelease
> Get:9 https://mirrors.wikimedia.org/debian unstable InRelease [235 kB]
> Get:10 https://mirrors.wikimedia.org/debian unstable/contrib Sources [50.8 kB]
> Get:11 https://mirrors.wikimedia.org/debian unstable/non-free Sources [83.8 
> kB]
> Get:12 https://mirrors.wikimedia.org/debian unstable/main Sources [7,848 kB]
> Get:13 https://mirrors.wikimedia.org/debian unstable/main amd64 Packages 
> [7,777 kB]
> Get:14 https://mirrors.wikimedia.org/debian unstable/main Translation-en 
> [5,867 kB]
> Get:15 https://mirrors.wikimedia.org/debian unstable/main amd64 DEP-11 
> Metadata [3,107 kB]
> Get:16 https://mirrors.wikimedia.org/debian unstable/main DEP-11 64x64 Icons 
> [6,936 kB]
> Get:17 https://mirrors.wikimedia.org/debian unstable/contrib amd64 Packages 
> [58.5 kB]
> Get:18 https://mirrors.wikimedia.org/debian unstable/contrib Translation-en 
> [50.9 kB]
> Get:19 https://mirrors.wikimedia.org/debian unstable/contrib amd64 DEP-11 
> Metadata [2,924 B]
> Get:20 https://mirrors.wikimedia.org/debian unstable/contrib DEP-11 64x64 
> Icons [37.7 kB]
> Get:21 https://mirrors.wikimedia.org/debian unstable/non-free amd64 Packages 
> [81.9 kB]
> Get:22 https://mirrors.wikimedia.org/debian unstable/non-free Translation-en 
> [83.8 kB]
> Get:23 https://mirrors.wikimedia.org/debian unstable/non-free amd64 DEP-11 
> Metadata [6,384 B]
> Get:24 https://mirrors.wikimedia.org/debian unstable/non-free DEP-11 64x64 
> Icons [1,813 B]
> Err:23 https://mirrors.wikimedia.org/debian unstable/non-free amd64 DEP-11 
> Metadata
>   Hash Sum mismatch
>   Hashes of expected file:
>- Filesize:65721 [weak]
>- SHA256:0eb5ea70b4c46a7c0889e9601d16cedf3eb4ff55bf3b77ef2af2bcd3dcf48cec
>- MD5Sum:68fdfbc3bc806010ba4c9fc8d4cb133b [weak]
>   Hashes of received file:
>- SHA256:5fa520cd26eac94e183632f84371643219fdf905dfd351b53bbcd7afa5c3b848
>- MD5Sum:875dd6824d698bfa73c6517307f692f5 [weak]
>- Filesize:65721 [weak]
>   Last modification reported: Mon, 30 Oct 2017 21:09:03 +
>   Release file created at: Fri, 03 Nov 2017 02:41:19 +
> Reading package lists... Done   
> E: Failed to fetch 
> store:/var/lib/apt/lists/partial/mirrors.wikimedia.org_debian_dists_unstable_non-free_dep11_Components-amd64.yml.xz
>   Hash Sum mismatch
>Hashes of expected file:
> - Filesize:65721 [weak]
> - SHA256:0eb5ea70b4c46a7c0889e9601d16cedf3eb4ff55bf3b77ef2af2bcd3dcf48cec
> - MD5Sum:68fdfbc3bc806010ba4c9fc8d4cb133b [weak]
>Hashes of received file:
> - SHA256:5fa520cd26eac94e183632f84371643219fdf905dfd351b53bbcd7afa5c3b848
> - MD5Sum:875dd6824d698bfa73c6517307f692f5 [weak]
> - Filesize:65721 [weak]
>Last modification reported: Mon, 30 Oct 2017 21:09:03 +
>Release file created at: Fri, 03 Nov 2017 02:41:19 +
> E: Some index files failed to download. They have been ignored, or old ones 
> used instead.
> ```
> 
The archive is broken, .xz contains different content than .gz - writing the 
.xz file
failed. The are several .xz.new files, so I expect some of the other files are 
broken
too.



-- 
Debian Developer - deb.li/jak | jak-linux.org - free software dev
Ubuntu Core Developer  de, en speaker