>Synopsis: relayd mixes up filter rules when a protocol is reused in
>multiple relays
>Category: system
>Environment:
System : OpenBSD 6.6
Details : OpenBSD 6.6 (GENERIC.MP) #372: Sat Oct 12 10:56:27 MDT
2019
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
Architecture: OpenBSD.amd64
Machine : amd64
>Description:
I set up relayd in front of httpd, so that http://$(hostname)/* should
redirect to
https://$(hostname)/*, https://$(hostname)/ is the main site but
https://$(hostname)/app is a
embedded webapp. This sort of reverse proxy set up is common with
webapps (like NextCloud,
Mattermost, Gitea, Radicale, etc) to avoid having to fight the
same-origin policy: it allows
/app and the main site to communicate via cookies, websockets, etc. It
also avoids firewalls
that otherwise block webapps run on unusual ports (e.g. the way cPanel
is usually installed)
by making all apps share a single port: 443.
But relayd applies the /app proxy I asked of it *to the http:// site*,
which means people can
get at my webapp without going through TLS.
# Afterthought
After working with this for a bit it's obvious to me that relayd was
not meant for doing this
kind of sub-domain/sub-path reverse proxying. What I really want is
httpd's location blocks,
which are simple and easy, and importantly feature the `strip` command,
but to be able to apply
them to a TCP proxy and not just to FastCGI.
>How-To-Repeat:
0) Make an empty test folder
```
$ mkdir t1; cd t1
```
1) Set up httpd with two vhosts plus a "bounce to https". The first vhost is
our main site, the second, running on a different port, represents a backend
webapp that runs its own HTTP server
```
$ mkdir -p site app
$ mkdir -p logs # httpd insists on this
$ echo "Static Site" > site/index.html
$ echo "WebApp" > app/index.html
$ cat > httpd.conf <<EOF
# httpd.conf
chroot "."
server "default" {
listen on localhost port 8080
location * {
block return 302 "https://\$HTTP_HOST\$REQUEST_URI";
}
}
server "site" {
listen on localhost port 8081
root "site"
directory auto index
}
server "app" {
listen on localhost port 8082
root "app"
request strip 1 # this site will be mounted at /app but relayd doesn't
have (obvious?) URL rewriting
directory auto index
}
EOF
```
```
$ doas httpd -f httpd.conf # httpd demands root, even though it doesn't need it
in this case
```
Test:
```
$ curl http://localhost:8081/
Static Site
$ curl http://localhost:8082/
WebApp
```
2) Make a cert for TLS
This will be used to set up relayd listening on two ports -- http and https --
and demonstrate
This makes a self-signed cert so it'll have to `curl --cacert $(hostname).pem`
(or just `curl -k`) to use it; if you have a live server you can experiment on,
use acme-client to get a "real" cert, or reuse a valid cert you have.
```
$ openssl req -x509 -newkey rsa:4096 -subj '/CN='"$(hostname)" -nodes -keyout
$(hostname).key -out $(hostname).pem -days 3
Generating a 4096 bit RSA private key
...................................................................++++
................................................................................................................................++++
writing new private key to 'matriculate.lan.key'
-----
```
# relayd config
3) Set up relayd with rules to rewrite the URLs.
```
$ cat > relayd.conf <<EOF
table <web> { "127.0.0.1" }
table <app> { "127.0.0.1" }
http protocol web {
# Return HTTP/HTML error pages to the client
return error
tls keypair $(hostname)
#match request path "/app{,/*}" tag app # want this, but this glob
syntax isn't supported?
match request path "/app" tag app
match request path "/app/*" tag app
match request tagged app header set "X-Script-Name" value "/app" # has
to match the above!
match request tagged app forward to <app>
}
# http:// should bounce to the "default" vhost, which should bounce the client
to https://
relay http_proxy {
listen on * port 80
protocol web
forward to <web> port 8080
}
# https:// should handle
relay https_proxy {
listen on * port 443 tls
protocol web
forward to <web> port 8081
forward to <app> port 8082
}
EOF
```
relayd insists you need the certs to be global:
```
$ doas cp $(pwd)/$(hostname).pem /etc/ssl/$(hostname).crt
$ doas cp $(pwd)/$(hostname).key /etc/ssl/private/$(hostname).key
```
# Testing
In one shell:
```
$ doas relayd -d -v -f relayd.conf
startup
adding 1 hosts from table web:8080 (no check)
adding 1 hosts from table web:8080 (no check)
adding 1 hosts from table web:8080 (no check)
adding 1 hosts from table web:8081 (no check)
adding 1 hosts from table web:8081 (no check)
adding 1 hosts from table app:8082 (no check)
adding 1 hosts from table app:8082 (no check)
adding 1 hosts from table web:8081 (no check)
adding 1 hosts from table app:8082 (no check)
```
in a parallel shell:
a) demonstrate that http://$(hostname)/, https://$(hostname)/ and
https://$(hostname)/app are working as expected
```
$ curl -v http://$(hostname)/
* Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to matriculate.lan (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: matriculate.lan
> User-Agent: curl/7.66.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
* HTTP 1.0, assume close after body
< HTTP/1.0 302 Found
< Connection: close
< Content-Length: 419
< Content-Type: text/html
< Date: Mon, 10 Feb 2020 02:19:00 GMT
< Location: https://matriculate.lan/
< Server: OpenBSD httpd
<
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
<title>302 Found</title>
<style type="text/css"><!--
body { background-color: white; color: black; font-family: 'Comic Sans MS',
'Chalkboard SE', 'Comic Neue', sans-serif; }
hr { border: 0; border-bottom: 1px dashed; }
--></style>
</head>
<body>
<h1>302 Found</h1>
<hr>
<address>OpenBSD httpd</address>
</body>
</html>
* Closing connection 0
$ curl --cacert /etc/ssl/$(hostname).crt https://$(hostname)/
Static Site
$ curl --cacert /etc/ssl/$(hostname).crt https://$(hostname)/app/
WebApp
```
b) But http://$(hostname)/app is *wrong*: it passes through directly to the
backend without hitting the 302 Redirect:
```
$ curl http://$(hostname)/app/
WebApp
```
The WebApp, like the Static site, are supposed to be *inaccessible* over http.
So what went wrong here?
Appendix: repro v2 (without TLS)
--------------------------------
The bug doesn't depend on TLS, it was just a bit more realistic and motivating
to demonstrate it that way.
It still appears if we replace https://$(hostname)/ with
http://$(hostname):8080/
```
$ mkdir t2; cd t2
$ mkdir -p site app
$ mkdir -p logs # httpd insists on this
$ echo "Static Site" > site/index.html
$ echo "WebApp" > app/index.html
$
$ cat > httpd.conf <<EOF
chroot "."
server "site" {
listen on localhost port 8081
root "site"
directory auto index
}
server "app" {
listen on localhost port 8082
root "app"
request strip 1
directory auto index
}
EOF
$ doas httpd -f httpd.conf
$
$ cat > relayd.conf <<EOF
table <web> { "127.0.0.1" }
table <app> { "127.0.0.1" }
http protocol web {
# Return HTTP/HTML error pages to the client
return error
#match request path "/app{,/*}" tag app # want this, but this glob
syntax isn't supported?
match request path "/app" tag app
match request path "/app/*" tag app
match request tagged app forward to <app>
}
relay http_proxy {
listen on 0.0.0.0 port 80
protocol web
forward to <web> port 8081
}
relay http_alt_proxy {
listen on 0.0.0.0 port 8080
protocol web
forward to <web> port 8081
forward to <app> port 8082
}
EOF
$ doas relayd -f relayd.conf
```
Then:
```
$ curl http://$(hostname)/app/
WebApp
```
even though the port 80 relay isn't supposed to be forwarding to the webapp.
If we add more "webapps":
```
$ mkdir app2; echo "WebApp 2" > app2/index.html
$ cat >> httpd.conf <EOF
server "app2" {
listen on localhost port 8083
root "app2"
request strip 1
directory auto index
}
EOF
$ vi relayd.conf # ... edit to say:
$ cat relayd.conf
table <web> { "127.0.0.1" }
table <app> { "127.0.0.1" }
table <app2> { "127.0.0.1" }
http protocol web {
# Return HTTP/HTML error pages to the client
return error
#match request path "/app{,/*}" tag app # want this, but this glob
syntax isn't supported?
match request path "/app" tag app
match request path "/app/*" tag app
match request tagged app forward to <app>
#match request path "/app2{,/*}" tag app2 # want this, but this glob
syntax isn't supported?
match request path "/app2" tag app2
match request path "/app2/*" tag app2
match request tagged app2 forward to <app2>
}
relay http_proxy {
listen on 0.0.0.0 port 80
protocol web
forward to <web> port 8081
}
relay http_alt_proxy {
listen on 0.0.0.0 port 8080
protocol web
forward to <web> port 8081
forward to <app> port 8082
forward to <app2> port 8083
}
$ doas pkill httpd
$ doas pkill relayd
$ doas httpd -f httpd.conf
$ doas relayd -f relayd.conf
```
The problem continues:
```
$ curl http://$(hostname):8080/ # expected
Static Site
$ curl http://$(hostname):8080/app/ # expected
WebApp
$ curl http://$(hostname):8080/app2/ # expected
WebApp 2
$
$ curl http://$(hostname)/ # expected
Static Site
$ curl http://$(hostname)/app/ # BUG
WebApp
$ curl http://$(hostname)/app2/ # BUG
WebApp 2
```
>Fix:
Adding this to relayd.conf:
```
http protocol web80 {
return error
}
```
and making `relay http_proxy` use `protocol web80` makes the problem go away.
But if you can't reuse protocols why distinguish them from relays at all?
