Yes, I should have explained the problem up front. I made the wrong assumption
I was asking a simple question and quickly realized I was getting good answers
but my approach was likely flawed from the start.
We are using Nginx just as a cache mechanism across many custom DNS names. As a
result we have many server {} blocks handling a variety of different DNS names,
vanity URLs and in a few cases SSL redirects. Where possible a single server {}
block handles a wildcard DNS name but there are reasons we separate some of
them out. Given the number of DNS entries, I have also created a system for
creating *.conf files, so I do not use a single conf file for all the servers,
instead I divide them out into many conf files. For special configuration items
that are only included sometimes, I use *.inc files with the idea being that
nginx.conf includes *.conf but does not include *.inc so that way I can make my
server{} blocks cleaner and put common configurations into inc file. The main
benefit being that almost all the config files fit on the screen without
scrolling that much in VIM.
Within each of these DNS instances we have a mix of login required content. We
use three different login methods, IP whitelists, local user/password set in
our CMS and then we use NetID via CAS/Shibboleth. We don’t want any of the
login pages cached.
The way we differentiate this relates to the mechanics on the backend of the
web application. So we created a custom header called X-Secured-Page which is
either true or false. In the event the value is true, we do not want to cache.
To go a step further, I have made use the add_header to set different headers
for when we cache a page versus when we do not cache a page. The idea being
that Tier1 support could view these headers to deflect support calls when
things are working as intended but also to create Jira tickets with the custom
header content when things do not seem to be working correctly. In addition to
this, when the secure page is true, I use this setting proxy_pass_header
Set-Cookie; but when the secured page is false I use this setting:
proxy_hide_header Set-Cookie; As for why, it is actually the CAS/Shibboleth
authentication that fails to work without this setting change. I could easily
just always pass the cookie but I was trying to eliminate any noise I could. We
have a fairly complicated stack given the Nginx passes most traffic to a server
cluster that handles URL rewrites and Vanity URLs at a level we did not want to
approach in Nginx. We may in the future but right now that is off the table
given the complexity and work that went into the management interfaces used to
handle the url rewrites the content owners use.
So my thought process started with a test for $http_x_secured_page which for
obvious reasons would not work because I was failing to distinguish request
versus response headers and am very new to Nginx.
Using the responses yesterday, I was 99% sure I had a working model nailed
down. I had a map block that looked like this
Map $upstream_http_x_secured_page $nocache {
default “0”;
“true” “1”;
“false” “2”;
}
Then in my server block I had logic that looks like this
Server{
Location / {
If ($nocache = “1”) { return 418; }
If ($nocache = “2”) { return 419; };
}
}
Then I had a custom location for each, again this was all just to test and it
worked. I used a combination of cURL and just using the inspector in firefox to
see the headers and I was consistently getting the correct code from either the
418 or 419 block. So I moved the same identical code into production but used a
new server block. In all this testing I was working with a DNS name not yet put
into Nginx, so I was using all HOSTS file manipulation. But the content of the
Server block was accurate so the responses were correct. This is the same
method I used to test every DNS name moved in and seemed like the simplest way
to test in production, not to mention we run 4 nginx servers as part of a
globally redundant solution, using the HOSTS file allows me to easily test
against a single production server before rolling my configs to all servers.
In trying to figure out the problem, I read about how the map{} block worked
and was confused by it saying that it stores the last request made which caused
me to wonder if the design I had with many server{} blocks all sharing the same
map{} directive was somehow flawed so my testing in production was doomed from
the start. I also really do not understand the order of operation taking place.
If the map{} took place inside a server block, then I would understand it
better and see it encapsulated from other server{} requests. Or if I was to use
upstream{} blocks and it went inside that, again I would understand it more.
I did try implementing upstream{} sections but it did not seem to make sense, I
only had one server defined inside of it and since I did not see a direct
relationship with map{} and in testing it made no difference whether I had this
defined or not, so I removed it.
The bottom line is that I need a solution that will cache almost all pages and
then in some cases when the X-Secured-Page is true, I don’t want to cache it. I
am totally flexible, we could easily set it up so that the X-Secured-Page
header only exists when the value would be true and does not exist at all when
it would be false if that would make things easier or more reliable. I’m also
open to doing just about anything else to make this happen.
On the code side, this value is determined in a security class that was already
setting some headings we used for debugging at the application level and it
would not be hard to setup a service that could be called (even cached behind
nginx) where a URL is cURL’d to see if the page should be secure or not. I’m
not sure if having that sort of logic inside a server{} block is possible, good
practice or the stupidest idea in the world.
But like I said, I’m willing to approach this problem however necessary with
one exception. As it stands now, I have a server block that we moved in which
is when we realized there was going to be a problem with login pages. So inside
the server{} block I setup location{} blocks for the folders that were secure
and handled them with the alternative add_header and cookie settings. Security
on folders can be a little more dynamic than I’d like and also not predictable.
We have hundreds of web content editors that can make secure content easily in
our CMS and they make significant use of this option. So creating a system that
would use location{} blocks has to be off the table. It would be a tremendous
amount of work to maintain that setup.
___________________________________________
Michael Friscia
Office of Communications
Yale School of Medicine
(203) 737-7932 - office
(203) 931-5381 - mobile
http://web.yale.edu <http://web.yale.edu/>
On 2/14/18, 5:13 PM, "nginx on behalf of Peter Booth" <nginx-boun...@nginx.org
on behalf of peter_bo...@me.com> wrote:
I think that part of the power and challenge of using nginx’s caching is
that there are many different ways
of achieving the same or similar results, but some of the approaches will
be more awkward than others.
I think that it might help if you could express what the issue is that you
are trying to solve,
as opposed to the mechanism that you are wanting to use to solve your
problem, for example,
“Don’t cache personalized pages” or “don’t cache pages where the user is
outside the US” or
“don’t cache pages if the user has logged in” or ...
> On Feb 14, 2018, at 4:44 PM, Friscia, Michael <michael.fris...@yale.edu>
wrote:
>
> Maybe that’s the problem, I want to disable cache if the response header
is true but not do anything if it is false. I can change my logic in creating
this header to only have it on pages where cache should be disabled if it is
not possible to use an IF statement around it. I will post my config here once
I get rid of the sensitive things, but the confusing thing is that it worked,
then it stopped working.
>
>
>
> ___________________________________________
>
> Michael Friscia
>
> Office of Communications
> Yale School of Medicine
>
> (203) 737-7932 – office
> (203) 931-5381 – mobile
>
> http://web.yale.edu
>
>
>
>
> On 2/14/18, 4:39 PM, "nginx on behalf of webopsx"
<nginx-boun...@nginx.org on behalf of nginx-fo...@forum.nginx.org> wrote:
>
> Hi,
>
> The map is processed on each request and should be very consistent.
>
> I thought you wanted to disable cache on the existence of a response
header,
> not a request header.
>
> Otherwise I think we need more information to understand, such as how
are
> you testing? Perhaps paste your full configuration here after removing
any
> sensitive text.
>
> Posted at Nginx Forum:
https://urldefense.proofpoint.com/v2/url?u=https-3A__forum.nginx.org_read.php-3F2-2C278558-2C278576-23msg-2D278576&d=DwICAg&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=x0V3h5hp-I18hCxDQuN0U7f_JYUQYISVWlygl1QK-FU&s=sUMR29LFjCI97P-LSb-RwuojlvhtDhvfRiX_YAio19E&e=
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
>
https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.nginx.org_mailman_listinfo_nginx&d=DwICAg&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=x0V3h5hp-I18hCxDQuN0U7f_JYUQYISVWlygl1QK-FU&s=833NZxJsLHFd1YlQmOtXp22oR4K0OXPS-xz0mYs6r04&e=
>
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
>
https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.nginx.org_mailman_listinfo_nginx&d=DwIGaQ&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=JuKVbxbm0g2sxo5YSYeiKyCXurdeZujyxenVsST6Ce8&s=m6rC0_mW4r0GP-Yl6edLAFDbcVLuRViJIOg-r3yQ9vc&e=
_______________________________________________
nginx mailing list
nginx@nginx.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.nginx.org_mailman_listinfo_nginx&d=DwIGaQ&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=JuKVbxbm0g2sxo5YSYeiKyCXurdeZujyxenVsST6Ce8&s=m6rC0_mW4r0GP-Yl6edLAFDbcVLuRViJIOg-r3yQ9vc&e=
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
