[PATCH] Support HTTP/2 headers from web servers.

[Derek Upham]

The web client module doesn’t currently support HTTPS, so I have been calling 
out to curl and parsing the result with the web client code.  This is breaking 
for HTTP/2 servers, as the code expects a MAJOR.MINOR format, and the HTTP/2 
working group is explicitly not using a minor version in the HTTP/2 spec 
(https://http2.github.io/faq/#is-it-http20-or-http2).  For example,

  curl --silent --head https://www.google.com

returns the first line

  HTTP/2 200

This patch adds parsing and printing support that handles HTTP/2, representing 
it as a consistent ‘'(2 . 0)’ internally.

I’m tempted to drop the whole “find the dot index” approach for HTTP/1.x and 
just explicitly match on “HTTP/1.0” and “HTTP/1.1” as well.

Derek

>From f957bd2b496a2aea92d1184e57c18215acc3ed22 Mon Sep 17 00:00:00 2001
From: Derek Upham 
Date: Tue, 4 Jul 2017 08:54:06 -0700
Subject: [PATCH] Support HTTP/2 headers from web servers.

For example,

  curl --silent --head https://www.google.com

returns the first line

  HTTP/2 200

which does not match the MAJOR.MINOR format expected by the old code.
---
 module/web/http.scm | 41 ++---
 1 file changed, 26 insertions(+), 15 deletions(-)

diff --git a/module/web/http.scm b/module/web/http.scm
index 993b50ef4..db0dff76d 100644
--- a/module/web/http.scm
+++ b/module/web/http.scm
@@ -1059,7 +1059,11 @@ as an ordered alist."
 (define* (parse-http-version str #:optional (start 0) (end (string-length str)))
   "Parse an HTTP version from STR, returning it as a major–minor
 pair. For example, ‘HTTP/1.1’ parses as the pair of integers,
-‘(1 . 1)’."
+‘(1 . 1)’.
+
+The HTTP/2 working group has explicitly dropped the ‘.0’ minor
+version.  For internal consistency, track the the missing minor
+version as zero."
   (let lp ((known *known-versions*))
 (match known
   (((version-str . version-val) . known)
@@ -1067,28 +1071,35 @@ pair. For example, ‘HTTP/1.1’ parses as the pair of integers,
version-val
(lp known)))
   (()
-   (let ((dot-idx (string-index str #\. start end)))
- (unless (and (string-prefix? "HTTP/" str 0 5 start end)
-  dot-idx
-  (= dot-idx (string-rindex str #\. start end)))
-   
-   (bad-header-component 'http-version (substring str start end)))
- (cons (parse-non-negative-integer str (+ start 5) dot-idx)
-   (parse-non-negative-integer str (1+ dot-idx) end)))
+   (if (string=? "HTTP/2" (substring str start end))
+   '(2 . 0)
+   (let ((dot-idx (string-index str #\. start end)))
+ (unless (and (string-prefix? "HTTP/" str 0 5 start end)
+  dot-idx
+  (= dot-idx (string-rindex str #\. start end)))
+   (bad-header-component 'http-version (substring str start end)))
+ (cons (parse-non-negative-integer str (+ start 5) dot-idx)
+   (parse-non-negative-integer str (1+ dot-idx) end
 
 (define (write-http-version val port)
-  "Write the given major-minor version pair to PORT."
-  (put-string port "HTTP/")
-  (put-non-negative-integer port (car val))
-  (put-char port #\.)
-  (put-non-negative-integer port (cdr val)))
+  "Write the given major-minor version pair to PORT.
+
+For consistency with HTTP/2 standards, print version ‘(2 . 0)’ as
+“HTTP/2”."
+  (if (equal? val '(2 . 0))
+  (put-string port "HTTP/2")
+  (begin
+(put-string port "HTTP/")
+(put-non-negative-integer port (car val))
+(put-char port #\.)
+(put-non-negative-integer port (cdr val)
 
 (for-each
  (lambda (v)
(set! *known-versions*
  (acons v (parse-http-version v 0 (string-length v))
 *known-versions*)))
- '("HTTP/1.0" "HTTP/1.1"))
+ '("HTTP/1.0" "HTTP/1.1" "HTTP/2"))
 
 
 ;; Request-URI = "*" | absoluteURI | abs_path | authority
-- 
2.13.1


-- 
Derek Upham
s...@blarg.net

Adding (ice-9 suspendable-ports) support to https / custom ports

[Christopher Allan Webber]

Hiya,

For the project I'm working on, I'll need to have (ice-9
suspendable-ports) work with https.  This is a bit more urgent than I
realized.  I'm willing to attempt the work, but I don't even know where
to start.

Could someone give me pointers as to where to begin spelunking?  I'm
guessing I have to both follow the lead of the suspendable-ports
adjustments to normal ports and also enable fnctl support for custom
ports?  Specifically, I imagine that this needs to be supported:

 (let ((flags (fcntl socket F_GETFL)))
   (fcntl socket F_SETFL (logior O_NONBLOCK flags)))

Either that or we need some generalized procedure that can either do
this to a port which does set fnctl or... I don't know what it would do
for something which doesn't need to run that operation.

Advice appreciated!
 - Chris