Thanks for the help! For posterity, here is the moral equivalent of the relayd.conf that is working for me:
table <httpd> { 127.0.0.1 }
table <app1> { 127.0.0.1 }
http protocol "https" {
tls { keypair "example.com" }
match request header "Host" value "app1.example.com" \
forward to <app1>
}
# Terminate tls and proxy to httpd
relay "tlsproxy" {
listen on egress port 443 tls
protocol "https"
forward to <httpd> port 8000
forward to <app1> port 9991
}

