Re: Allow internal redirect to URI x, but deny external request for x?

2019-09-10 Thread j94305
Robots exclusion is generally quite unreliable. Exclusions based on user agents are also not really reliable. You can try all of the options for robots exclusion and may still get undesired crawlers on your site. The only way you can keep robots out is to require authentication for those parts

Re: Allow internal redirect to URI x, but deny external request for x?

2019-09-04 Thread J. Lewis Muir
On 09/04, Jürgen Wagner (DVT) wrote: > This is the effect you get by having the HTTP equivalent of a symbolic link > in the NGINX (visible to the browser), not in the file system (which is > opaque to users). The file system link will (over time) serve different > contents under the same URL, so

Re: Allow internal redirect to URI x, but deny external request for x?

2019-09-04 Thread DVT
Hi Lewis,   no, that won't cause double requests. /myapp/current/blah.html 307 => /myapp/releases/1.2.0/blah.html and from thereon (as we did not redirect internally, but rather externally), any further accesses will happen unter the true "releases" path (ideally, as relative URLs).

Re: Allow internal redirect to URI x, but deny external request for x?

2019-09-04 Thread J. Lewis Muir
On 09/04, Jürgen Wagner (DVT) wrote: > Now, you want to be able to say what is the "current" version and reflect > this in the URL namespace as well. In the file system, that's a symbolic > link. In the URL namespace of NGINX, that could be a redirection (status > code 307). Both approaches would

Re: Allow internal redirect to URI x, but deny external request for x?

2019-09-03 Thread DVT
Hi Lewis,   the idea is to have a deployment process that places apps or whatever artifacts always in a certain distinct place that is determined once at deployment time. This will determine the address where you can reach the app in the namespace of NGINX. So, if the convention is to place

Re: Allow internal redirect to URI x, but deny external request for x?

2019-09-03 Thread J. Lewis Muir
On 08/30, j94305 wrote: > I've been following this, and I would take a slightly different approach. > > 1. Serve all apps under /{app}/releases/{version}/{path} as you have them > organized in the deployment structure in the file system. > > 2. Forget about symbolic links and other makeshift

Re: Allow internal redirect to URI x, but deny external request for x?

2019-09-03 Thread J. Lewis Muir
On 09/03, J. Lewis Muir wrote: > On 09/02, Francis Daly wrote: > > But if "the app" involves a http request to part1.php and then a http > > request to part2.php (or: a second http request to part1.php), I don't > > think that the symlink+realpath thing will prevent those two requests > > going to

Re: Allow internal redirect to URI x, but deny external request for x?

2019-09-03 Thread J. Lewis Muir
On 09/02, Francis Daly wrote: > nginx does not "do" php. nginx does not care what your fastcgi server > will do with the key/value pairs that it sends. nginx cares that the > fastcgi server gives a valid response to the request that nginx makes. > > Typically, your fastcgi server will use the

Re: Allow internal redirect to URI x, but deny external request for x?

2019-09-02 Thread Francis Daly
On Sat, Aug 31, 2019 at 04:55:26PM -0500, J. Lewis Muir wrote: > On 08/31, Francis Daly wrote: > > On Sat, Aug 31, 2019 at 09:10:09AM -0500, J. Lewis Muir wrote: Hi there, > > Using "realpath" should not affect nginx at all. nginx invites the > > fastcgi server to use pathname2 instead of

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-31 Thread J. Lewis Muir
On 08/31, Francis Daly wrote: > On Sat, Aug 31, 2019 at 09:10:09AM -0500, J. Lewis Muir wrote: > > On 08/31, Francis Daly wrote: > > Hi there, > > > > * starts with /my-app/current/ -> reject > > > * starts with /my-app/releases/ -> reject > > Actually -- those two "rejects" should not be

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-31 Thread Francis Daly
On Sat, Aug 31, 2019 at 09:10:09AM -0500, J. Lewis Muir wrote: > On 08/31, Francis Daly wrote: Hi there, > > * starts with /my-app/current/ -> reject > > * starts with /my-app/releases/ -> reject Actually -- those two "rejects" should not be needed. The app probably should not be installed

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-31 Thread lists
I don't run WordPress. I found a list of bad user agents on GitHub that I flag on.   Original Message   From: hobso...@gmail.com Sent: August 31, 2019 7:41 AM To: nginx@nginx.org Reply-to: nginx@nginx.org Subject: Re: Allow internal redirect to URI x, but deny external request for x? Hi

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-31 Thread J. Lewis Muir
On 08/30, Ian Hobson wrote: > Hi Lewis, > > On 30/08/19 18:33, J. Lewis Muir wrote: > > Hello! > > > > I'm using nginx 1.12.2 on RHEL 7, and I've got a FastCGI web app that > > uses a deployment structure which uses an atomic symlink change for an > > atomic app deploy, and I'm wishing to be

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-31 Thread J. Lewis Muir
On 08/31, Francis Daly wrote: > On Sat, Aug 31, 2019 at 12:21:40AM +0100, Francis Daly wrote: > > Hi there, > > A few further thoughts here... > > > It sounds like your desires are for requests: > > > > * starts with /my-app/current/ -> reject > > * starts with /my-app/releases/ -> reject >

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-31 Thread Ian Hobson
al Message From: hobso...@gmail.com Sent: August 30, 2019 12:01 PM To: nginx@nginx.org Reply-to: nginx@nginx.org Subject: Re: Allow internal redirect to URI x, but deny external request for x? Hi Lewis, On 30/08/19 18:33, J. Lewis Muir wrote: Hello! I'm using nginx 1.12.2 on RHEL 7, and I've got

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-31 Thread J. Lewis Muir
On 08/31, Francis Daly wrote: > On Fri, Aug 30, 2019 at 04:59:36PM -0500, J. Lewis Muir wrote: > > Hi there, > > > I was wishing for a way to specify a new root but with a modified > > request URI. So, I tried the alias directive, and I assumed that > > $document_root and $realpath_root would

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-31 Thread Francis Daly
On Sat, Aug 31, 2019 at 12:21:40AM +0100, Francis Daly wrote: Hi there, A few further thoughts here... > It sounds like your desires are for requests: > > * starts with /my-app/current/ -> reject > * starts with /my-app/releases/ -> reject > * matches /my-app/something.php, or

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-30 Thread Francis Daly
On Fri, Aug 30, 2019 at 04:59:36PM -0500, J. Lewis Muir wrote: Hi there, > I was wishing for a way to specify a new root but with a modified > request URI. So, I tried the alias directive, and I assumed that > $document_root and $realpath_root would refer to the aliased document > root, but

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-30 Thread j94305
I've been following this, and I would take a slightly different approach. 1. Serve all apps under /{app}/releases/{version}/{path} as you have them organized in the deployment structure in the file system. 2. Forget about symbolic links and other makeshift versioning/defaulting in the file

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-30 Thread J. Lewis Muir
On 08/30, Francis Daly wrote: > On Fri, Aug 30, 2019 at 01:58:23PM -0500, J. Lewis Muir wrote: > > Hi there, > > > location ~ ^/my-app/(.*?[^/]\.php(?:/.*|$)) { > > alias /srv/www/my-app/current/$1; > > fastcgi_split_path_info ^(.+?\.php)(/.*)$; > > return 200 "realpath_root:

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-30 Thread lists
nt: August 30, 2019 12:01 PM To: nginx@nginx.org Reply-to: nginx@nginx.org Subject: Re: Allow internal redirect to URI x, but deny external request for x? Hi Lewis, On 30/08/19 18:33, J. Lewis Muir wrote: > Hello! > > I'm using nginx 1.12.2 on RHEL 7, and I've got a FastCGI web app that

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-30 Thread Francis Daly
On Fri, Aug 30, 2019 at 01:58:23PM -0500, J. Lewis Muir wrote: Hi there, > location ~ ^/my-app/(.*?[^/]\.php(?:/.*|$)) { > alias /srv/www/my-app/current/$1; > fastcgi_split_path_info ^(.+?\.php)(/.*)$; > return 200 "realpath_root: $realpath_root\nfastcgi_script_name: >

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-30 Thread Francis Daly
On Fri, Aug 30, 2019 at 12:33:17PM -0500, J. Lewis Muir wrote: Hi there, > I'm wishing to be able to do an internal redirect > in nginx to URL x, but deny an external request to the same URL x so > that I don't serve the same content at more than one URL. Is there a > way to do that? > > For

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-30 Thread Ian Hobson
Hi Lewis, On 30/08/19 18:33, J. Lewis Muir wrote: Hello! I'm using nginx 1.12.2 on RHEL 7, and I've got a FastCGI web app that uses a deployment structure which uses an atomic symlink change for an atomic app deploy, and I'm wishing to be able to do an internal redirect in nginx to URL x, but

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-30 Thread J. Lewis Muir
On 08/30, J. Lewis Muir wrote: > On 08/30, J. Lewis Muir wrote: > > I'm wondering if the fastcgi_split_path_info function or the if > > directive is what's emitting the file op message in the error log which > > would mean that one or both of $realpath_root or $fastcgi_script_name > > are not set

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-30 Thread J. Lewis Muir
On 08/30, J. Lewis Muir wrote: > I'm wondering if the fastcgi_split_path_info function or the if > directive is what's emitting the file op message in the error log which > would mean that one or both of $realpath_root or $fastcgi_script_name > are not set to what I expect. (?) Adding return

Re: Allow internal redirect to URI x, but deny external request for x?

2019-08-30 Thread J. Lewis Muir
On 08/30, J. Lewis Muir wrote: > I initially tried using the alias directive which I thought was a better > fit for what I wanted to do > > location /my-app/ { > alias /srv/www/my-app/current/; > index index.php; > } > > location /my-app/current/ { > return 404; > } > >

Allow internal redirect to URI x, but deny external request for x?

2019-08-30 Thread J. Lewis Muir
Hello! I'm using nginx 1.12.2 on RHEL 7, and I've got a FastCGI web app that uses a deployment structure which uses an atomic symlink change for an atomic app deploy, and I'm wishing to be able to do an internal redirect in nginx to URL x, but deny an external request to the same URL x so that I