[Bug-wget] [PATCH 25/25] New: --metalink-over-http Content-Type/Disposition Metalink/XML processing

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This implements the auto-processing of Metalink files part of a "Content-Type: application/metalink4+xml" header answer. e.g.: $ wget --metalink-over-http \ --metalink-index=inf \ [

[Bug-wget] [PATCH 22/25] Bugfix: Detect when a metalink:file doesn't have any hash

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] Before this patch, a Metalink/XML metalink:file that doesn't provide any hash was causing a segmentation fault. This properly handles the situation when a Metalink/XML metalink:file doesn't prov

[Bug-wget] [PATCH 24/25] Bugfix: Set NULL variable due to --content-disposition to Metalink origin

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This solves a segmentation fault when local_file is not properly set, i.e. due to the use of the option --content-disposition. The segmentation fault was generated trying to dereference a NULL lo

[Bug-wget] [PATCH 21/25] Bugfix: Detect malformed base64 Metalink/HTTP Digest header

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] Before this patch, a malformed base64 Metalink/HTTP Digest header was causing a memory exaustion. This introduces the ability to properly detect and discard a malformed base64 Metalink/HTTP Dige

[Bug-wget] [PATCH 23/25] New: --trust-server-names saves Metalink/HTTP xml files using the "name" field

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This allows to use the value of the "name" field, part of a Metalink/HTTP application/metalink4+xml Link, as file name for the Metalink file to be saved. This conforms to the RFC6249 standard: ht

[Bug-wget] [PATCH 20/25] New option --metalink-index to process Metalink application/metalink4+xml

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This introduces the new option --metalink-index=NUMBER, for auto-processing of Metalink files contained in a Metalink/HTTP header answer. --metalink-index= saves the user from manually (cli) 1. c

[Bug-wget] [PATCH 19/25] Bugfix: Prevent sorting of unallocated metalink resource/metaurl

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This ensures that Metalink resources/metaurls are sorted only when necessary, preventing useless sorting and possible segmentation faults. The following description is verbatim from the patch: --

[Bug-wget] [PATCH 18/25] New: Parse Metalink/HTTP header for application/metalink4+xml

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, contrib/check-hard is ok] `make check-valgrind` fails the following metalink tests due to the bugs which will be fixed in '[PATCH 19/25] Bugfix: Prevent sorting of unallocated metalink resource/metaurl': * testenv/Test-metalink-http.py * testenv/Tes

[Bug-wget] [PATCH 14/25] New test: Detect when there are no good Metalink url resources

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This adds a new Metalink test. The following description is verbatim from the patch: - Test if when there are no good Metalink url resources there is any segmentation fault. - Regards, Ma

[Bug-wget] [PATCH 15/25] Bugfix: Process Metalink/XML url strings containing white spaces and CRLF

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This fixes problems introduced by processing a Metalink/XML file containing metalink:url strings separated by CRLF. The following description is verbatim from the patch: - White spaces and CR

[Bug-wget] [PATCH 17/25] New test: Metalink shall not concatenate '/' to an empty directory prefix

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This adds a new Metalink test. The following description is verbatim from the patch: - Detect a '/' character wrongfully concatenated to an empty directory prefix '' (not NULL), resulting in a

[Bug-wget] [PATCH 16/25] Bugfix: Remove surrounding quotes from Metalink/HTTP key's value

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This fixes problems introduced by processing a Metalink/HTTP header which contains quoted values. The following description is verbatim from the patch: - Some Metalink/HTTP keys, like "type"

[Bug-wget] [PATCH 12/25] New test: --continue shall keep fully retrieved Metalink files (HTTP 416)

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This introduces a new Metalink test to verify that when --continue is used, after a HTTP 416 answer a file that fails a sanity test isn't renamed/removed. The following description is verbatim fr

[Bug-wget] [PATCH 13/25] New: Metalink file size mismatch returns error code METALINK_SIZE_ERROR

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This adds a specific error code to use when a Metalink file fails its size verification. The following description is verbatim from the patch: - Before this patch, no appropriate error code w

[Bug-wget] [PATCH 11/25] New: Metalink/XML and Metalink/HTTP file naming safety rules

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This introduces new rules/tests about Metalink/XML and Metalink/HTTP. The safety mechanism introduced provides secure and predictable file names. This is convenient to prevent the overwriting of

[Bug-wget] [PATCH 10/25] New document: Metalink/XML and Metalink/HTTP standard reference

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This introduces the new document doc/metalink-standard.txt. The purpose of the document is to serve as a Metalink/XML and Metalink/HTTP standard reference of the Metalink functionalities currentl

[Bug-wget] [PATCH 09/25] Enforce Metalink file name verification, strip directory if necessary

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This enforces the use of libmetalink's metalink_check_safe_path() to verify that the destination file name is safe. The libmetalink's metalink_check_safe_path() advices against the use of unsafe

[Bug-wget] [PATCH 08/25] Implement Metalink/XML --directory-prefix option in Metalink module

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This implements the use of the option --directory-prefix in the Metalink module. The following description is verbatim from the patch: - When --directory-prefix= is used, set the top of the re

[Bug-wget] [PATCH 07/25] Change mfile->name to filename in Metalink module's messages

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This fixes the downloaded/saved file name used when printing the output. The following description is verbatim from the patch: - The file name could have been changed by unique_create() (or by

[Bug-wget] [PATCH 06/25] Add file size computation in Metalink module

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok] [The contrib/check-hard warnings are at the bottom] This adds the file size computation feature to the Metalink module. The following description is verbatim from the patch: - Reject downloaded files when they do not a

[Bug-wget] [PATCH 04/25] Bugfix: Keep the download progress when alternating metalink:url

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This serves a double purpose: - when a Metalink resource fails, keep the download progress and continue with the next resource; - when --continue is used, retain the existing file or the just full

[Bug-wget] [PATCH 05/25] Update Metalink/XML tests and add a new test for home paths

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This updates some Metalink tests and add one test: * testenv/Test-metalink-xml-relpath.py: Reject relative paths * testenv/Test-metalink-xml-homepath.py: New file. Reject home paths The following

[Bug-wget] [PATCH 02/25] Add metalink description

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] Add doc/metalink.txt to describe the behaviour of the GNU Wget Metalink module. The doc/metalink.txt file will be progressively updated to describe new features and/or different behaviours. Rega

[Bug-wget] [PATCH 03/25] Bugfix: Fix NULL filename and output_stream in Metalink module

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] This fixes some problems leading to a possible segmentation fault, and conforms to the RFC5854 standard. The following description is verbatim from the patch: - If unique_create() cannot crea

[Bug-wget] [PATCH 01/25] Add two Metalink/XML tests

2016-09-10 Thread Matthew White
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, contrib/check-hard is ok] Patch provided by Tim, slightly amended to make the tests to return a proper result that conforms to the RFC5854 standard [1]. References: [1] https://tools.ietf.org/html/rfc5854#section-4.1.2.1