Re: [Bug-wget] [PATCH 23/25] New: --trust-server-names saves Metalink/HTTP xml files using the "name" field

2016-09-11 Thread Giuseppe Scrivano 
Hi Matthew,

Matthew White  writes:

> [Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, 
> contrib/check-hard is ok]
>
> This allows to use the value of the "name" field, part of a Metalink/HTTP 
> application/metalink4+xml Link, as file name for the Metalink file to be 
> saved.
>
> This conforms to the RFC6249 standard:
> https://tools.ietf.org/html/rfc6249#section-4
>
> e.g.
> $ wget --metalink-over-http --metalink-index=inf --trust-server-names  a Metalink/HTTP header with application/metlaink4+xml links>
>
>   Link: ; rel=describedby; 
> type="application/metalink4+xml"; name="newname.meta4"
>
>   When --trust-server-names is used, file.ext.meta4 is saved as newname.meta4.
>
> Regards,
> Matthew
>
> -- 
> Matthew White 
>
> From 30b09eb7dbfbb9423efb6bd4009203f887d90786 Mon Sep 17 00:00:00 2001
> From: Matthew White 
> Date: Tue, 30 Aug 2016 09:26:41 +0200
> Subject: [PATCH 23/25] New: --trust-server-names saves Metalink/HTTP xml files
>  using the "name" field
>
> * src/metalink.c (retrieve_from_metalink): If opt.trustservernames is
>   true, use the basename of the metaurl's name to save the xml file
> * doc/metalink-standard.txt: Update doc. With --trust-server-names any
>   Metalink/HTTP Link application/metalink4+xml file is saved using the
>   basename of the "name" field, if any. Update Metalink/HTTP examples
> * testenv/Makefile.am: Add new file
> * testenv/Test-metalink-http-xml-trust-name.py: New file. Metalink/HTTP
>   automated Metalink/XML, save xml files using the "name" field tests
> ---
>  doc/metalink-standard.txt|  10 +
>  src/metalink.c   |   2 +-
>  testenv/Makefile.am  |   1 +
>  testenv/Test-metalink-http-xml-trust-name.py | 273 
> +++
>  4 files changed, 285 insertions(+), 1 deletion(-)
>  create mode 100755 testenv/Test-metalink-http-xml-trust-name.py

ACK

Giuseppe

[Bug-wget] [PATCH 23/25] New: --trust-server-names saves Metalink/HTTP xml files using the "name" field

2016-09-10 Thread Matthew White 
[Coverity Scan is ok, make syntax-check is ok, make check-valgrind is ok, 
contrib/check-hard is ok]

This allows to use the value of the "name" field, part of a Metalink/HTTP 
application/metalink4+xml Link, as file name for the Metalink file to be saved.

This conforms to the RFC6249 standard:
https://tools.ietf.org/html/rfc6249#section-4

e.g.
$ wget --metalink-over-http --metalink-index=inf --trust-server-names 

  Link: ; rel=describedby; 
type="application/metalink4+xml"; name="newname.meta4"

  When --trust-server-names is used, file.ext.meta4 is saved as newname.meta4.

Regards,
Matthew

-- 
Matthew White 
>From 30b09eb7dbfbb9423efb6bd4009203f887d90786 Mon Sep 17 00:00:00 2001
From: Matthew White 
Date: Tue, 30 Aug 2016 09:26:41 +0200
Subject: [PATCH 23/25] New: --trust-server-names saves Metalink/HTTP xml files
 using the "name" field

* src/metalink.c (retrieve_from_metalink): If opt.trustservernames is
  true, use the basename of the metaurl's name to save the xml file
* doc/metalink-standard.txt: Update doc. With --trust-server-names any
  Metalink/HTTP Link application/metalink4+xml file is saved using the
  basename of the "name" field, if any. Update Metalink/HTTP examples
* testenv/Makefile.am: Add new file
* testenv/Test-metalink-http-xml-trust-name.py: New file. Metalink/HTTP
  automated Metalink/XML, save xml files using the "name" field tests
---
 doc/metalink-standard.txt|  10 +
 src/metalink.c   |   2 +-
 testenv/Makefile.am  |   1 +
 testenv/Test-metalink-http-xml-trust-name.py | 273 +++
 4 files changed, 285 insertions(+), 1 deletion(-)
 create mode 100755 testenv/Test-metalink-http-xml-trust-name.py

diff --git a/doc/metalink-standard.txt b/doc/metalink-standard.txt
index 78709fb..4836a85 100644
--- a/doc/metalink-standard.txt
+++ b/doc/metalink-standard.txt
@@ -83,6 +83,9 @@ When --trust-server-names is on, the metalink:file "name" field parsed
 from Metalink/XML files is trusted. When no Metalink/XML is available,
 the mother URL is trusted.
 
+Any Metalink/HTTP application/metalink4+xml file is saved using the
+basename of its own Link header "name" field, if available.
+
 3.1.2 The final name
 
 
@@ -183,6 +186,13 @@ type="application/pgp-signature"
 Digest: SHA-256=MWVkMWQxYTRiMzk5MDQ0MzI3NGU5NDEyZTk5OWY1ZGFmNzgyZTJlO
 DYzYjRjYzFhOTlmNTQwYzI2M2QwM2U2MQ==
 
+See [2 #section-4].
+
+Link: ; rel=describedby;
+type="application/x-bittorrent"; name="differentname.ext"
+Link: ; rel=describedby;
+type="application/metalink4+xml"
+
 5.4 Saving files
 
 
diff --git a/src/metalink.c b/src/metalink.c
index 1506825..21fef67 100644
--- a/src/metalink.c
+++ b/src/metalink.c
@@ -240,7 +240,7 @@ retrieve_from_metalink (const metalink_t* metalink)
 metafile = xstrdup (safename);
 
 if (opt.trustservernames)
-  replace_metalink_basename (&metafile, murl->url);
+  replace_metalink_basename (&metafile, murl->name ? murl->name : murl->url);
 else
   append_suffix_number (&metafile, ".meta#", meta_count);
 
diff --git a/testenv/Makefile.am b/testenv/Makefile.am
index 2cd5ff5..0ef0915 100644
--- a/testenv/Makefile.am
+++ b/testenv/Makefile.am
@@ -32,6 +32,7 @@ if METALINK_IS_ENABLED
 Test-metalink-http-baddigest.py \
 Test-metalink-http-xml.py   \
 Test-metalink-http-xml-trust.py \
+Test-metalink-http-xml-trust-name.py\
 Test-metalink-xml.py\
 Test-metalink-xml-relpath.py\
 Test-metalink-xml-abspath.py\
diff --git a/testenv/Test-metalink-http-xml-trust-name.py b/testenv/Test-metalink-http-xml-trust-name.py
new file mode 100755
index 000..69e633e
--- /dev/null
+++ b/testenv/Test-metalink-http-xml-trust-name.py
@@ -0,0 +1,273 @@
+#!/usr/bin/env python3
+from sys import exit
+from test.http_test import HTTPTest
+from misc.wget_file import WgetFile
+import re
+import hashlib
+from base64 import b64encode
+
+"""
+This is to test Metalink/HTTP with Metalink/XML Link headers.
+
+With --trust-server-names, trust the metalink:file names.
+
+Without --trust-server-names, don't trust the metalink:file names:
+use the basename of --input-metalink, and add a sequential number
+(e.g. .#1, .#2, etc.).
+
+Strip the directory from unsafe paths.
+"""
+
+# File Definitions ###
+bad = "Ouch!"
+bad_sha256 = hashlib.sha256 (bad.encode ('UTF-8')).hexdigest ()
+
+File1 = "Would you like some Tea?"
+File1_lowPref = "Do not take this"
+File1_sha256 = hashlib.sha256 (File1.encode ('UTF-8')).hexdigest ()
+
+File2 = "This is gonna be good"
+File2_lowPref