Re: Httpd support for internal redirects.

[Ori Bernstein]

On Sat, 21 Oct 2017 22:18:17 +0200, Anders Andersson  wrote:

> Maybe I don't fully understand the problem, but is it not better to
> fix your broken website generator than to add more code to httpd?
> 

It seemed like other people wanted the feature as well, and based
on the contents of the cited thread, it seemed like something that
might be accepted if implemented.

It was easy to implement, so I added it and sent a patch. If
it's not a desirable feature for httpd, no problem.

-- 
Ori Bernstein

Re: Httpd support for internal redirects.

[Anders Andersson]

On Wed, Oct 11, 2017 at 6:31 AM, Ori Bernstein  wrote:
> My website generator is a little stupid at times. It generates
> files with .html suffixes, but urls without them.
>
> I worked around this with some redirects, but it never felt
> quite right doing an extra round trip. Therefore, I added
> internal redirects, processing the rewrite before responding to
> an http request.

Maybe I don't fully understand the problem, but is it not better to
fix your broken website generator than to add more code to httpd?

Re: Httpd support for internal redirects.

[Gregory Edigarov]

as a, well, co-author of some earlier attempt on this, the developers 
just do not interested in this.
so we are left on our own to maintain this. as a side note my diff is 
more functional then your's
as it provides a way to test  the existence of the filesystem object 
before rewrite happen.





On 19.10.17 00:39, Ori Bernstein wrote:

Pinging this patch.

On Tue, 10 Oct 2017 21:31:20 -0700
Ori Bernstein  wrote:


My website generator is a little stupid at times. It generates
files with .html suffixes, but urls without them.

I worked around this with some redirects, but it never felt
quite right doing an extra round trip. Therefore, I added
internal redirects, processing the rewrite before responding to
an http request.

This introduces new syntax to the config file, allowing you to
do:

location match "^(/foo/bar/[%w]+)$" {
rewrite-to "/baz/%1.html"
}

Because we don't know what the paths should be relative
to, all paths rewritten must be absolute.

It seems like someone else may find it useful[1], so
I'm submitting it. I've been running a slightly older
version of this on https://myrlang.org for the last
day or two, and it's been uneventful. The difference
is that the syntax used to piggy back off the "block"
action => 'block internal return 302 "path"'.

This doesn't currently support chained rewrites. I think
that it wouldn't be hard to add if it's needed.

Ok?

[1] https://github.com/reyk/httpd/issues/27
==


diff --git usr.sbin/httpd/config.c usr.sbin/httpd/config.c
index 3c31c3d4cd3..7d2982af7b9 100644
--- usr.sbin/httpd/config.c
+++ usr.sbin/httpd/config.c
@@ -448,7 +448,7 @@ config_getserver_config(struct httpd *env, struct server 
*srv,
sizeof(srv_conf->errorlog));
}
  
-		f = SRVFLAG_BLOCK|SRVFLAG_NO_BLOCK;

+   f = SRVFLAG_BLOCK|SRVFLAG_NO_BLOCK|SRVFLAG_REWRITE;
if ((srv_conf->flags & f) == 0) {
free(srv_conf->return_uri);
srv_conf->flags |= parent->flags & f;
diff --git usr.sbin/httpd/httpd.conf.5 usr.sbin/httpd/httpd.conf.5
index a3c97629de3..3a00a750537 100644
--- usr.sbin/httpd/httpd.conf.5
+++ usr.sbin/httpd/httpd.conf.5
@@ -454,6 +454,14 @@ instead of the log files.
  Disable any previous
  .Ic block
  in a location.
+.It Ic rewrite-to Ar path
+The current request path is rewritten to
+.Ar  path .
+using the same macro expansions as
+.Cm block return
+rules. After macros are substituted, the resulting paths must be
+absolute, beginning with a slash.  Rewriting is not done
+recursively.
  .It Ic root Ar option
  Configure the document root and options for the request path.
  Valid options are:
diff --git usr.sbin/httpd/httpd.h usr.sbin/httpd/httpd.h
index 05cbb8e3550..477115ec92d 100644
--- usr.sbin/httpd/httpd.h
+++ usr.sbin/httpd/httpd.h
@@ -394,6 +394,7 @@ SPLAY_HEAD(client_tree, client);
  #define SRVFLAG_SERVER_MATCH  0x0020
  #define SRVFLAG_SERVER_HSTS   0x0040
  #define SRVFLAG_DEFAULT_TYPE  0x0080
+#define SRVFLAG_REWRITE0x0100
  
  #define SRVFLAG_BITS			\

"\10\01INDEX\02NO_INDEX\03AUTO_INDEX\04NO_AUTO_INDEX" \
diff --git usr.sbin/httpd/parse.y usr.sbin/httpd/parse.y
index fcf1938c42d..4072ee5b532 100644
--- usr.sbin/httpd/parse.y
+++ usr.sbin/httpd/parse.y
@@ -134,7 +134,7 @@ typedef struct {
  %tokenLISTEN LOCATION LOG LOGDIR MATCH MAXIMUM NO NODELAY OCSP ON 
PORT PREFORK
  %tokenPROTOCOLS REQUESTS ROOT SACK SERVER SOCKET STRIP STYLE SYSLOG 
TCP TICKET
  %tokenTIMEOUT TLS TYPE TYPES HSTS MAXAGE SUBDOMAINS DEFAULT PRELOAD 
REQUEST
-%token ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN PASS
+%token ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN REWRITE PASS
  %tokenSTRING
  %tokenNUMBER
  %type   port
@@ -986,6 +986,11 @@ filter : block RETURN NUMBER optstring {
srv_conf->return_uri_len = strlen($4) + 1;
}
}
+   | REWRITE STRING {
+   srv_conf->flags |= SRVFLAG_REWRITE;
+   srv_conf->return_uri = $2;
+   srv_conf->return_uri_len = strlen($2) + 1;
+   }
| block DROP{
/* No return code, silently drop the connection */
srv_conf->return_code = 0;
@@ -1255,6 +1260,7 @@ lookup(char *s)
{ "request",  REQUEST },
{ "requests", REQUESTS },
{ "return",   RETURN },
+   { "rewrite-to",   REWRITE },
{ "root", ROOT },
{ "sack", SACK },
{ "server",   SERVER },
diff --git usr.sbin/httpd/server_http.c usr.sbin/httpd/server_http.c
index 

Re: Httpd support for internal redirects.

[Ori Bernstein]

Pinging this patch.

On Tue, 10 Oct 2017 21:31:20 -0700
Ori Bernstein  wrote:

> My website generator is a little stupid at times. It generates
> files with .html suffixes, but urls without them.
> 
> I worked around this with some redirects, but it never felt
> quite right doing an extra round trip. Therefore, I added
> internal redirects, processing the rewrite before responding to
> an http request.
> 
> This introduces new syntax to the config file, allowing you to
> do:
> 
>   location match "^(/foo/bar/[%w]+)$" {
>   rewrite-to "/baz/%1.html"
>   }
> 
> Because we don't know what the paths should be relative
> to, all paths rewritten must be absolute.
> 
> It seems like someone else may find it useful[1], so
> I'm submitting it. I've been running a slightly older
> version of this on https://myrlang.org for the last
> day or two, and it's been uneventful. The difference
> is that the syntax used to piggy back off the "block"
> action => 'block internal return 302 "path"'.
> 
> This doesn't currently support chained rewrites. I think
> that it wouldn't be hard to add if it's needed.
> 
> Ok?
> 
> [1] https://github.com/reyk/httpd/issues/27
> ==
> 
> 
> diff --git usr.sbin/httpd/config.c usr.sbin/httpd/config.c
> index 3c31c3d4cd3..7d2982af7b9 100644
> --- usr.sbin/httpd/config.c
> +++ usr.sbin/httpd/config.c
> @@ -448,7 +448,7 @@ config_getserver_config(struct httpd *env, struct server 
> *srv,
>   sizeof(srv_conf->errorlog));
>   }
>  
> - f = SRVFLAG_BLOCK|SRVFLAG_NO_BLOCK;
> + f = SRVFLAG_BLOCK|SRVFLAG_NO_BLOCK|SRVFLAG_REWRITE;
>   if ((srv_conf->flags & f) == 0) {
>   free(srv_conf->return_uri);
>   srv_conf->flags |= parent->flags & f;
> diff --git usr.sbin/httpd/httpd.conf.5 usr.sbin/httpd/httpd.conf.5
> index a3c97629de3..3a00a750537 100644
> --- usr.sbin/httpd/httpd.conf.5
> +++ usr.sbin/httpd/httpd.conf.5
> @@ -454,6 +454,14 @@ instead of the log files.
>  Disable any previous
>  .Ic block
>  in a location.
> +.It Ic rewrite-to Ar path
> +The current request path is rewritten to
> +.Ar  path .
> +using the same macro expansions as
> +.Cm block return
> +rules. After macros are substituted, the resulting paths must be
> +absolute, beginning with a slash.  Rewriting is not done
> +recursively.
>  .It Ic root Ar option
>  Configure the document root and options for the request path.
>  Valid options are:
> diff --git usr.sbin/httpd/httpd.h usr.sbin/httpd/httpd.h
> index 05cbb8e3550..477115ec92d 100644
> --- usr.sbin/httpd/httpd.h
> +++ usr.sbin/httpd/httpd.h
> @@ -394,6 +394,7 @@ SPLAY_HEAD(client_tree, client);
>  #define SRVFLAG_SERVER_MATCH 0x0020
>  #define SRVFLAG_SERVER_HSTS  0x0040
>  #define SRVFLAG_DEFAULT_TYPE 0x0080
> +#define SRVFLAG_REWRITE  0x0100
>  
>  #define SRVFLAG_BITS \
>   "\10\01INDEX\02NO_INDEX\03AUTO_INDEX\04NO_AUTO_INDEX"   \
> diff --git usr.sbin/httpd/parse.y usr.sbin/httpd/parse.y
> index fcf1938c42d..4072ee5b532 100644
> --- usr.sbin/httpd/parse.y
> +++ usr.sbin/httpd/parse.y
> @@ -134,7 +134,7 @@ typedef struct {
>  %token   LISTEN LOCATION LOG LOGDIR MATCH MAXIMUM NO NODELAY OCSP ON 
> PORT PREFORK
>  %token   PROTOCOLS REQUESTS ROOT SACK SERVER SOCKET STRIP STYLE SYSLOG 
> TCP TICKET
>  %token   TIMEOUT TLS TYPE TYPES HSTS MAXAGE SUBDOMAINS DEFAULT PRELOAD 
> REQUEST
> -%token   ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN PASS
> +%token   ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN REWRITE PASS
>  %token STRING
>  %token NUMBER
>  %typeport
> @@ -986,6 +986,11 @@ filter   : block RETURN NUMBER optstring {
>   srv_conf->return_uri_len = strlen($4) + 1;
>   }
>   }
> + | REWRITE STRING {
> + srv_conf->flags |= SRVFLAG_REWRITE;
> + srv_conf->return_uri = $2;
> + srv_conf->return_uri_len = strlen($2) + 1;
> + }
>   | block DROP{
>   /* No return code, silently drop the connection */
>   srv_conf->return_code = 0;
> @@ -1255,6 +1260,7 @@ lookup(char *s)
>   { "request",REQUEST },
>   { "requests",   REQUESTS },
>   { "return", RETURN },
> + { "rewrite-to", REWRITE },
>   { "root",   ROOT },
>   { "sack",   SACK },
>   { "server", SERVER },
> diff --git usr.sbin/httpd/server_http.c usr.sbin/httpd/server_http.c
> index e64de0d2f9c..c9ea4771037 100644
> --- usr.sbin/httpd/server_http.c
> +++ usr.sbin/httpd/server_http.c
> @@ -1162,10 +1162,34 @@ 

Httpd support for internal redirects.

[Ori Bernstein]

My website generator is a little stupid at times. It generates
files with .html suffixes, but urls without them.

I worked around this with some redirects, but it never felt
quite right doing an extra round trip. Therefore, I added
internal redirects, processing the rewrite before responding to
an http request.

This introduces new syntax to the config file, allowing you to
do:

location match "^(/foo/bar/[%w]+)$" {
rewrite-to "/baz/%1.html"
}

Because we don't know what the paths should be relative
to, all paths rewritten must be absolute.

It seems like someone else may find it useful[1], so
I'm submitting it. I've been running a slightly older
version of this on https://myrlang.org for the last
day or two, and it's been uneventful. The difference
is that the syntax used to piggy back off the "block"
action => 'block internal return 302 "path"'.

This doesn't currently support chained rewrites. I think
that it wouldn't be hard to add if it's needed.

Ok?

[1] https://github.com/reyk/httpd/issues/27
==


diff --git usr.sbin/httpd/config.c usr.sbin/httpd/config.c
index 3c31c3d4cd3..7d2982af7b9 100644
--- usr.sbin/httpd/config.c
+++ usr.sbin/httpd/config.c
@@ -448,7 +448,7 @@ config_getserver_config(struct httpd *env, struct server 
*srv,
sizeof(srv_conf->errorlog));
}
 
-   f = SRVFLAG_BLOCK|SRVFLAG_NO_BLOCK;
+   f = SRVFLAG_BLOCK|SRVFLAG_NO_BLOCK|SRVFLAG_REWRITE;
if ((srv_conf->flags & f) == 0) {
free(srv_conf->return_uri);
srv_conf->flags |= parent->flags & f;
diff --git usr.sbin/httpd/httpd.conf.5 usr.sbin/httpd/httpd.conf.5
index a3c97629de3..3a00a750537 100644
--- usr.sbin/httpd/httpd.conf.5
+++ usr.sbin/httpd/httpd.conf.5
@@ -454,6 +454,14 @@ instead of the log files.
 Disable any previous
 .Ic block
 in a location.
+.It Ic rewrite-to Ar path
+The current request path is rewritten to
+.Ar  path .
+using the same macro expansions as
+.Cm block return
+rules. After macros are substituted, the resulting paths must be
+absolute, beginning with a slash.  Rewriting is not done
+recursively.
 .It Ic root Ar option
 Configure the document root and options for the request path.
 Valid options are:
diff --git usr.sbin/httpd/httpd.h usr.sbin/httpd/httpd.h
index 05cbb8e3550..477115ec92d 100644
--- usr.sbin/httpd/httpd.h
+++ usr.sbin/httpd/httpd.h
@@ -394,6 +394,7 @@ SPLAY_HEAD(client_tree, client);
 #define SRVFLAG_SERVER_MATCH   0x0020
 #define SRVFLAG_SERVER_HSTS0x0040
 #define SRVFLAG_DEFAULT_TYPE   0x0080
+#define SRVFLAG_REWRITE0x0100
 
 #define SRVFLAG_BITS   \
"\10\01INDEX\02NO_INDEX\03AUTO_INDEX\04NO_AUTO_INDEX"   \
diff --git usr.sbin/httpd/parse.y usr.sbin/httpd/parse.y
index fcf1938c42d..4072ee5b532 100644
--- usr.sbin/httpd/parse.y
+++ usr.sbin/httpd/parse.y
@@ -134,7 +134,7 @@ typedef struct {
 %token LISTEN LOCATION LOG LOGDIR MATCH MAXIMUM NO NODELAY OCSP ON PORT PREFORK
 %token PROTOCOLS REQUESTS ROOT SACK SERVER SOCKET STRIP STYLE SYSLOG TCP TICKET
 %token TIMEOUT TLS TYPE TYPES HSTS MAXAGE SUBDOMAINS DEFAULT PRELOAD REQUEST
-%token ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN PASS
+%token ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN REWRITE PASS
 %token   STRING
 %token   NUMBER
 %type  port
@@ -986,6 +986,11 @@ filter : block RETURN NUMBER optstring {
srv_conf->return_uri_len = strlen($4) + 1;
}
}
+   | REWRITE STRING {
+   srv_conf->flags |= SRVFLAG_REWRITE;
+   srv_conf->return_uri = $2;
+   srv_conf->return_uri_len = strlen($2) + 1;
+   }
| block DROP{
/* No return code, silently drop the connection */
srv_conf->return_code = 0;
@@ -1255,6 +1260,7 @@ lookup(char *s)
{ "request",REQUEST },
{ "requests",   REQUESTS },
{ "return", RETURN },
+   { "rewrite-to", REWRITE },
{ "root",   ROOT },
{ "sack",   SACK },
{ "server", SERVER },
diff --git usr.sbin/httpd/server_http.c usr.sbin/httpd/server_http.c
index e64de0d2f9c..c9ea4771037 100644
--- usr.sbin/httpd/server_http.c
+++ usr.sbin/httpd/server_http.c
@@ -1162,10 +1162,34 @@ server_expand_http(struct client *clt, const char *val, 
char *buf,
return (buf);
 }
 
+static int
+server_set_path(struct http_descriptor *desc, char *input)
+{
+   char path[PATH_MAX];
+
+   if (input == NULL || url_decode(input) == NULL)
+   return -1;
+   if