Re: Httpd support for internal redirects.
On Sat, 21 Oct 2017 22:18:17 +0200, Anders Andersson wrote: > Maybe I don't fully understand the problem, but is it not better to > fix your broken website generator than to add more code to httpd? > It seemed like other people wanted the feature as well, and based on the contents of the cited thread, it seemed like something that might be accepted if implemented. It was easy to implement, so I added it and sent a patch. If it's not a desirable feature for httpd, no problem. -- Ori Bernstein
Re: Httpd support for internal redirects.
On Wed, Oct 11, 2017 at 6:31 AM, Ori Bernstein wrote: > My website generator is a little stupid at times. It generates > files with .html suffixes, but urls without them. > > I worked around this with some redirects, but it never felt > quite right doing an extra round trip. Therefore, I added > internal redirects, processing the rewrite before responding to > an http request. Maybe I don't fully understand the problem, but is it not better to fix your broken website generator than to add more code to httpd?
Re: Httpd support for internal redirects.
as a, well, co-author of some earlier attempt on this, the developers just do not interested in this. so we are left on our own to maintain this. as a side note my diff is more functional then your's as it provides a way to test the existence of the filesystem object before rewrite happen. On 19.10.17 00:39, Ori Bernstein wrote: Pinging this patch. On Tue, 10 Oct 2017 21:31:20 -0700 Ori Bernstein wrote: My website generator is a little stupid at times. It generates files with .html suffixes, but urls without them. I worked around this with some redirects, but it never felt quite right doing an extra round trip. Therefore, I added internal redirects, processing the rewrite before responding to an http request. This introduces new syntax to the config file, allowing you to do: location match "^(/foo/bar/[%w]+)$" { rewrite-to "/baz/%1.html" } Because we don't know what the paths should be relative to, all paths rewritten must be absolute. It seems like someone else may find it useful[1], so I'm submitting it. I've been running a slightly older version of this on https://myrlang.org for the last day or two, and it's been uneventful. The difference is that the syntax used to piggy back off the "block" action => 'block internal return 302 "path"'. This doesn't currently support chained rewrites. I think that it wouldn't be hard to add if it's needed. Ok? [1] https://github.com/reyk/httpd/issues/27 == diff --git usr.sbin/httpd/config.c usr.sbin/httpd/config.c index 3c31c3d4cd3..7d2982af7b9 100644 --- usr.sbin/httpd/config.c +++ usr.sbin/httpd/config.c @@ -448,7 +448,7 @@ config_getserver_config(struct httpd *env, struct server *srv, sizeof(srv_conf->errorlog)); } - f = SRVFLAG_BLOCK|SRVFLAG_NO_BLOCK; + f = SRVFLAG_BLOCK|SRVFLAG_NO_BLOCK|SRVFLAG_REWRITE; if ((srv_conf->flags & f) == 0) { free(srv_conf->return_uri); srv_conf->flags |= parent->flags & f; diff --git usr.sbin/httpd/httpd.conf.5 usr.sbin/httpd/httpd.conf.5 index a3c97629de3..3a00a750537 100644 --- usr.sbin/httpd/httpd.conf.5 +++ usr.sbin/httpd/httpd.conf.5 @@ -454,6 +454,14 @@ instead of the log files. Disable any previous .Ic block in a location. +.It Ic rewrite-to Ar path +The current request path is rewritten to +.Ar path . +using the same macro expansions as +.Cm block return +rules. After macros are substituted, the resulting paths must be +absolute, beginning with a slash. Rewriting is not done +recursively. .It Ic root Ar option Configure the document root and options for the request path. Valid options are: diff --git usr.sbin/httpd/httpd.h usr.sbin/httpd/httpd.h index 05cbb8e3550..477115ec92d 100644 --- usr.sbin/httpd/httpd.h +++ usr.sbin/httpd/httpd.h @@ -394,6 +394,7 @@ SPLAY_HEAD(client_tree, client); #define SRVFLAG_SERVER_MATCH 0x0020 #define SRVFLAG_SERVER_HSTS 0x0040 #define SRVFLAG_DEFAULT_TYPE 0x0080 +#define SRVFLAG_REWRITE0x0100 #define SRVFLAG_BITS \ "\10\01INDEX\02NO_INDEX\03AUTO_INDEX\04NO_AUTO_INDEX" \ diff --git usr.sbin/httpd/parse.y usr.sbin/httpd/parse.y index fcf1938c42d..4072ee5b532 100644 --- usr.sbin/httpd/parse.y +++ usr.sbin/httpd/parse.y @@ -134,7 +134,7 @@ typedef struct { %tokenLISTEN LOCATION LOG LOGDIR MATCH MAXIMUM NO NODELAY OCSP ON PORT PREFORK %tokenPROTOCOLS REQUESTS ROOT SACK SERVER SOCKET STRIP STYLE SYSLOG TCP TICKET %tokenTIMEOUT TLS TYPE TYPES HSTS MAXAGE SUBDOMAINS DEFAULT PRELOAD REQUEST -%token ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN PASS +%token ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN REWRITE PASS %tokenSTRING %tokenNUMBER %type port @@ -986,6 +986,11 @@ filter : block RETURN NUMBER optstring { srv_conf->return_uri_len = strlen($4) + 1; } } + | REWRITE STRING { + srv_conf->flags |= SRVFLAG_REWRITE; + srv_conf->return_uri = $2; + srv_conf->return_uri_len = strlen($2) + 1; + } | block DROP{ /* No return code, silently drop the connection */ srv_conf->return_code = 0; @@ -1255,6 +1260,7 @@ lookup(char *s) { "request", REQUEST }, { "requests", REQUESTS }, { "return", RETURN }, + { "rewrite-to", REWRITE }, { "root", ROOT }, { "sack", SACK }, { "server", SERVER }, diff --git usr.sbin/httpd/server_http.c usr.sbin/httpd/server_http.c index e64de0d2f9c..c9ea4771037 100644 --- usr.sbin/
Re: Httpd support for internal redirects.
Pinging this patch. On Tue, 10 Oct 2017 21:31:20 -0700 Ori Bernstein wrote: > My website generator is a little stupid at times. It generates > files with .html suffixes, but urls without them. > > I worked around this with some redirects, but it never felt > quite right doing an extra round trip. Therefore, I added > internal redirects, processing the rewrite before responding to > an http request. > > This introduces new syntax to the config file, allowing you to > do: > > location match "^(/foo/bar/[%w]+)$" { > rewrite-to "/baz/%1.html" > } > > Because we don't know what the paths should be relative > to, all paths rewritten must be absolute. > > It seems like someone else may find it useful[1], so > I'm submitting it. I've been running a slightly older > version of this on https://myrlang.org for the last > day or two, and it's been uneventful. The difference > is that the syntax used to piggy back off the "block" > action => 'block internal return 302 "path"'. > > This doesn't currently support chained rewrites. I think > that it wouldn't be hard to add if it's needed. > > Ok? > > [1] https://github.com/reyk/httpd/issues/27 > == > > > diff --git usr.sbin/httpd/config.c usr.sbin/httpd/config.c > index 3c31c3d4cd3..7d2982af7b9 100644 > --- usr.sbin/httpd/config.c > +++ usr.sbin/httpd/config.c > @@ -448,7 +448,7 @@ config_getserver_config(struct httpd *env, struct server > *srv, > sizeof(srv_conf->errorlog)); > } > > - f = SRVFLAG_BLOCK|SRVFLAG_NO_BLOCK; > + f = SRVFLAG_BLOCK|SRVFLAG_NO_BLOCK|SRVFLAG_REWRITE; > if ((srv_conf->flags & f) == 0) { > free(srv_conf->return_uri); > srv_conf->flags |= parent->flags & f; > diff --git usr.sbin/httpd/httpd.conf.5 usr.sbin/httpd/httpd.conf.5 > index a3c97629de3..3a00a750537 100644 > --- usr.sbin/httpd/httpd.conf.5 > +++ usr.sbin/httpd/httpd.conf.5 > @@ -454,6 +454,14 @@ instead of the log files. > Disable any previous > .Ic block > in a location. > +.It Ic rewrite-to Ar path > +The current request path is rewritten to > +.Ar path . > +using the same macro expansions as > +.Cm block return > +rules. After macros are substituted, the resulting paths must be > +absolute, beginning with a slash. Rewriting is not done > +recursively. > .It Ic root Ar option > Configure the document root and options for the request path. > Valid options are: > diff --git usr.sbin/httpd/httpd.h usr.sbin/httpd/httpd.h > index 05cbb8e3550..477115ec92d 100644 > --- usr.sbin/httpd/httpd.h > +++ usr.sbin/httpd/httpd.h > @@ -394,6 +394,7 @@ SPLAY_HEAD(client_tree, client); > #define SRVFLAG_SERVER_MATCH 0x0020 > #define SRVFLAG_SERVER_HSTS 0x0040 > #define SRVFLAG_DEFAULT_TYPE 0x0080 > +#define SRVFLAG_REWRITE 0x0100 > > #define SRVFLAG_BITS \ > "\10\01INDEX\02NO_INDEX\03AUTO_INDEX\04NO_AUTO_INDEX" \ > diff --git usr.sbin/httpd/parse.y usr.sbin/httpd/parse.y > index fcf1938c42d..4072ee5b532 100644 > --- usr.sbin/httpd/parse.y > +++ usr.sbin/httpd/parse.y > @@ -134,7 +134,7 @@ typedef struct { > %token LISTEN LOCATION LOG LOGDIR MATCH MAXIMUM NO NODELAY OCSP ON > PORT PREFORK > %token PROTOCOLS REQUESTS ROOT SACK SERVER SOCKET STRIP STYLE SYSLOG > TCP TICKET > %token TIMEOUT TLS TYPE TYPES HSTS MAXAGE SUBDOMAINS DEFAULT PRELOAD > REQUEST > -%token ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN PASS > +%token ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN REWRITE PASS > %token STRING > %token NUMBER > %typeport > @@ -986,6 +986,11 @@ filter : block RETURN NUMBER optstring { > srv_conf->return_uri_len = strlen($4) + 1; > } > } > + | REWRITE STRING { > + srv_conf->flags |= SRVFLAG_REWRITE; > + srv_conf->return_uri = $2; > + srv_conf->return_uri_len = strlen($2) + 1; > + } > | block DROP{ > /* No return code, silently drop the connection */ > srv_conf->return_code = 0; > @@ -1255,6 +1260,7 @@ lookup(char *s) > { "request",REQUEST }, > { "requests", REQUESTS }, > { "return", RETURN }, > + { "rewrite-to", REWRITE }, > { "root", ROOT }, > { "sack", SACK }, > { "server", SERVER }, > diff --git usr.sbin/httpd/server_http.c usr.sbin/httpd/server_http.c > index e64de0d2f9c..c9ea4771037 100644 > --- usr.sbin/httpd/server_http.c > +++ usr.sbin/httpd/server_http.c > @@ -1162,10 +1162,34 @@ server_expand_http(struct clien
Httpd support for internal redirects.
My website generator is a little stupid at times. It generates files with .html suffixes, but urls without them. I worked around this with some redirects, but it never felt quite right doing an extra round trip. Therefore, I added internal redirects, processing the rewrite before responding to an http request. This introduces new syntax to the config file, allowing you to do: location match "^(/foo/bar/[%w]+)$" { rewrite-to "/baz/%1.html" } Because we don't know what the paths should be relative to, all paths rewritten must be absolute. It seems like someone else may find it useful[1], so I'm submitting it. I've been running a slightly older version of this on https://myrlang.org for the last day or two, and it's been uneventful. The difference is that the syntax used to piggy back off the "block" action => 'block internal return 302 "path"'. This doesn't currently support chained rewrites. I think that it wouldn't be hard to add if it's needed. Ok? [1] https://github.com/reyk/httpd/issues/27 == diff --git usr.sbin/httpd/config.c usr.sbin/httpd/config.c index 3c31c3d4cd3..7d2982af7b9 100644 --- usr.sbin/httpd/config.c +++ usr.sbin/httpd/config.c @@ -448,7 +448,7 @@ config_getserver_config(struct httpd *env, struct server *srv, sizeof(srv_conf->errorlog)); } - f = SRVFLAG_BLOCK|SRVFLAG_NO_BLOCK; + f = SRVFLAG_BLOCK|SRVFLAG_NO_BLOCK|SRVFLAG_REWRITE; if ((srv_conf->flags & f) == 0) { free(srv_conf->return_uri); srv_conf->flags |= parent->flags & f; diff --git usr.sbin/httpd/httpd.conf.5 usr.sbin/httpd/httpd.conf.5 index a3c97629de3..3a00a750537 100644 --- usr.sbin/httpd/httpd.conf.5 +++ usr.sbin/httpd/httpd.conf.5 @@ -454,6 +454,14 @@ instead of the log files. Disable any previous .Ic block in a location. +.It Ic rewrite-to Ar path +The current request path is rewritten to +.Ar path . +using the same macro expansions as +.Cm block return +rules. After macros are substituted, the resulting paths must be +absolute, beginning with a slash. Rewriting is not done +recursively. .It Ic root Ar option Configure the document root and options for the request path. Valid options are: diff --git usr.sbin/httpd/httpd.h usr.sbin/httpd/httpd.h index 05cbb8e3550..477115ec92d 100644 --- usr.sbin/httpd/httpd.h +++ usr.sbin/httpd/httpd.h @@ -394,6 +394,7 @@ SPLAY_HEAD(client_tree, client); #define SRVFLAG_SERVER_MATCH 0x0020 #define SRVFLAG_SERVER_HSTS0x0040 #define SRVFLAG_DEFAULT_TYPE 0x0080 +#define SRVFLAG_REWRITE0x0100 #define SRVFLAG_BITS \ "\10\01INDEX\02NO_INDEX\03AUTO_INDEX\04NO_AUTO_INDEX" \ diff --git usr.sbin/httpd/parse.y usr.sbin/httpd/parse.y index fcf1938c42d..4072ee5b532 100644 --- usr.sbin/httpd/parse.y +++ usr.sbin/httpd/parse.y @@ -134,7 +134,7 @@ typedef struct { %token LISTEN LOCATION LOG LOGDIR MATCH MAXIMUM NO NODELAY OCSP ON PORT PREFORK %token PROTOCOLS REQUESTS ROOT SACK SERVER SOCKET STRIP STYLE SYSLOG TCP TICKET %token TIMEOUT TLS TYPE TYPES HSTS MAXAGE SUBDOMAINS DEFAULT PRELOAD REQUEST -%token ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN PASS +%token ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN REWRITE PASS %token STRING %token NUMBER %type port @@ -986,6 +986,11 @@ filter : block RETURN NUMBER optstring { srv_conf->return_uri_len = strlen($4) + 1; } } + | REWRITE STRING { + srv_conf->flags |= SRVFLAG_REWRITE; + srv_conf->return_uri = $2; + srv_conf->return_uri_len = strlen($2) + 1; + } | block DROP{ /* No return code, silently drop the connection */ srv_conf->return_code = 0; @@ -1255,6 +1260,7 @@ lookup(char *s) { "request",REQUEST }, { "requests", REQUESTS }, { "return", RETURN }, + { "rewrite-to", REWRITE }, { "root", ROOT }, { "sack", SACK }, { "server", SERVER }, diff --git usr.sbin/httpd/server_http.c usr.sbin/httpd/server_http.c index e64de0d2f9c..c9ea4771037 100644 --- usr.sbin/httpd/server_http.c +++ usr.sbin/httpd/server_http.c @@ -1162,10 +1162,34 @@ server_expand_http(struct client *clt, const char *val, char *buf, return (buf); } +static int +server_set_path(struct http_descriptor *desc, char *input) +{ + char path[PATH_MAX]; + + if (input == NULL || url_decode(input) == NULL) + return -1; + if (canonicalize_path(input