To add one more thing. I mentioned that my testing failed. Exactly what was
failing is that the map{} block that worked and then stopped working was the
problem, the $nocache variable would always return the default value no matter
what I did.
So in a previous post the suggested code was
proxy_no_cache $nocache;
I also included a add_header X-NoCacheValue $nocache;
My initial tests worked well, I saw the new header and the $nocache value that
was being decided was being passed just fine. But I don’t know if this sort of
use of the $nocache value was incorrect. I tried this using the IF() block I
mentioned below as well as eliminating the IF() blocks entirely and handling it
all within the server{} block to rule out problems with IF() which I am well
versed in.
___________________________________________
Michael Friscia
Office of Communications
Yale School of Medicine
(203) 737-7932 - office
(203) 931-5381 - mobile
http://web.yale.edu <http://web.yale.edu/>
On 2/15/18, 8:03 AM, "nginx on behalf of Friscia, Michael"
<nginx-boun...@nginx.org on behalf of michael.fris...@yale.edu> wrote:
Yes, I should have explained the problem up front. I made the wrong
assumption I was asking a simple question and quickly realized I was getting
good answers but my approach was likely flawed from the start.
We are using Nginx just as a cache mechanism across many custom DNS names.
As a result we have many server {} blocks handling a variety of different DNS
names, vanity URLs and in a few cases SSL redirects. Where possible a single
server {} block handles a wildcard DNS name but there are reasons we separate
some of them out. Given the number of DNS entries, I have also created a system
for creating *.conf files, so I do not use a single conf file for all the
servers, instead I divide them out into many conf files. For special
configuration items that are only included sometimes, I use *.inc files with
the idea being that nginx.conf includes *.conf but does not include *.inc so
that way I can make my server{} blocks cleaner and put common configurations
into inc file. The main benefit being that almost all the config files fit on
the screen without scrolling that much in VIM.
Within each of these DNS instances we have a mix of login required content.
We use three different login methods, IP whitelists, local user/password set in
our CMS and then we use NetID via CAS/Shibboleth. We don’t want any of the
login pages cached.
The way we differentiate this relates to the mechanics on the backend of
the web application. So we created a custom header called X-Secured-Page which
is either true or false. In the event the value is true, we do not want to
cache.
To go a step further, I have made use the add_header to set different
headers for when we cache a page versus when we do not cache a page. The idea
being that Tier1 support could view these headers to deflect support calls when
things are working as intended but also to create Jira tickets with the custom
header content when things do not seem to be working correctly. In addition to
this, when the secure page is true, I use this setting proxy_pass_header
Set-Cookie; but when the secured page is false I use this setting:
proxy_hide_header Set-Cookie; As for why, it is actually the CAS/Shibboleth
authentication that fails to work without this setting change. I could easily
just always pass the cookie but I was trying to eliminate any noise I could. We
have a fairly complicated stack given the Nginx passes most traffic to a server
cluster that handles URL rewrites and Vanity URLs at a level we did not want to
approach in Nginx. We may in the future but right now that is off the table
given the complexity and work that went into the management interfaces used to
handle the url rewrites the content owners use.
So my thought process started with a test for $http_x_secured_page which
for obvious reasons would not work because I was failing to distinguish request
versus response headers and am very new to Nginx.
Using the responses yesterday, I was 99% sure I had a working model nailed
down. I had a map block that looked like this
Map $upstream_http_x_secured_page $nocache {
default “0”;
“true” “1”;
“false” “2”;
}
Then in my server block I had logic that looks like this
Server{
Location / {
If ($nocache = “1”) { return 418; }
If ($nocache = “2”) { return 419; };
}
}
Then I had a custom location for each, again this was all just to test and
it worked. I used a combination of cURL and just using the inspector in firefox
to see the headers and I was consistently getting the correct code from either
the 418 or 419 block. So I moved the same identical code into production but
used a new server block. In all this testing I was working with a DNS name not
yet put into Nginx, so I was using all HOSTS file manipulation. But the content
of the Server block was accurate so the responses were correct. This is the
same method I used to test every DNS name moved in and seemed like the simplest
way to test in production, not to mention we run 4 nginx servers as part of a
globally redundant solution, using the HOSTS file allows me to easily test
against a single production server before rolling my configs to all servers.
In trying to figure out the problem, I read about how the map{} block
worked and was confused by it saying that it stores the last request made which
caused me to wonder if the design I had with many server{} blocks all sharing
the same map{} directive was somehow flawed so my testing in production was
doomed from the start. I also really do not understand the order of operation
taking place. If the map{} took place inside a server block, then I would
understand it better and see it encapsulated from other server{} requests. Or
if I was to use upstream{} blocks and it went inside that, again I would
understand it more.
I did try implementing upstream{} sections but it did not seem to make
sense, I only had one server defined inside of it and since I did not see a
direct relationship with map{} and in testing it made no difference whether I
had this defined or not, so I removed it.
The bottom line is that I need a solution that will cache almost all pages
and then in some cases when the X-Secured-Page is true, I don’t want to cache
it. I am totally flexible, we could easily set it up so that the X-Secured-Page
header only exists when the value would be true and does not exist at all when
it would be false if that would make things easier or more reliable. I’m also
open to doing just about anything else to make this happen.
On the code side, this value is determined in a security class that was
already setting some headings we used for debugging at the application level
and it would not be hard to setup a service that could be called (even cached
behind nginx) where a URL is cURL’d to see if the page should be secure or not.
I’m not sure if having that sort of logic inside a server{} block is possible,
good practice or the stupidest idea in the world.
But like I said, I’m willing to approach this problem however necessary
with one exception. As it stands now, I have a server block that we moved in
which is when we realized there was going to be a problem with login pages. So
inside the server{} block I setup location{} blocks for the folders that were
secure and handled them with the alternative add_header and cookie settings.
Security on folders can be a little more dynamic than I’d like and also not
predictable. We have hundreds of web content editors that can make secure
content easily in our CMS and they make significant use of this option. So
creating a system that would use location{} blocks has to be off the table. It
would be a tremendous amount of work to maintain that setup.
___________________________________________
Michael Friscia
Office of Communications
Yale School of Medicine
(203) 737-7932 - office
(203) 931-5381 - mobile
http://web.yale.edu <http://web.yale.edu/>
On 2/14/18, 5:13 PM, "nginx on behalf of Peter Booth"
<nginx-boun...@nginx.org on behalf of peter_bo...@me.com> wrote:
I think that part of the power and challenge of using nginx’s caching
is that there are many different ways
of achieving the same or similar results, but some of the approaches
will be more awkward than others.
I think that it might help if you could express what the issue is that
you are trying to solve,
as opposed to the mechanism that you are wanting to use to solve your
problem, for example,
“Don’t cache personalized pages” or “don’t cache pages where the user
is outside the US” or
“don’t cache pages if the user has logged in” or ...
> On Feb 14, 2018, at 4:44 PM, Friscia, Michael
<michael.fris...@yale.edu> wrote:
>
> Maybe that’s the problem, I want to disable cache if the response
header is true but not do anything if it is false. I can change my logic in
creating this header to only have it on pages where cache should be disabled if
it is not possible to use an IF statement around it. I will post my config here
once I get rid of the sensitive things, but the confusing thing is that it
worked, then it stopped working.
>
>
>
> ___________________________________________
>
> Michael Friscia
>
> Office of Communications
> Yale School of Medicine
>
> (203) 737-7932 – office
> (203) 931-5381 – mobile
>
> http://web.yale.edu
>
>
>
>
> On 2/14/18, 4:39 PM, "nginx on behalf of webopsx"
<nginx-boun...@nginx.org on behalf of nginx-fo...@forum.nginx.org> wrote:
>
> Hi,
>
> The map is processed on each request and should be very
consistent.
>
> I thought you wanted to disable cache on the existence of a
response header,
> not a request header.
>
> Otherwise I think we need more information to understand, such as
how are
> you testing? Perhaps paste your full configuration here after
removing any
> sensitive text.
>
> Posted at Nginx Forum:
https://urldefense.proofpoint.com/v2/url?u=https-3A__forum.nginx.org_read.php-3F2-2C278558-2C278576-23msg-2D278576&d=DwICAg&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=x0V3h5hp-I18hCxDQuN0U7f_JYUQYISVWlygl1QK-FU&s=sUMR29LFjCI97P-LSb-RwuojlvhtDhvfRiX_YAio19E&e=
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
>
https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.nginx.org_mailman_listinfo_nginx&d=DwICAg&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=x0V3h5hp-I18hCxDQuN0U7f_JYUQYISVWlygl1QK-FU&s=833NZxJsLHFd1YlQmOtXp22oR4K0OXPS-xz0mYs6r04&e=
>
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
>
https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.nginx.org_mailman_listinfo_nginx&d=DwIGaQ&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=JuKVbxbm0g2sxo5YSYeiKyCXurdeZujyxenVsST6Ce8&s=m6rC0_mW4r0GP-Yl6edLAFDbcVLuRViJIOg-r3yQ9vc&e=
_______________________________________________
nginx mailing list
nginx@nginx.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.nginx.org_mailman_listinfo_nginx&d=DwIGaQ&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=JuKVbxbm0g2sxo5YSYeiKyCXurdeZujyxenVsST6Ce8&s=m6rC0_mW4r0GP-Yl6edLAFDbcVLuRViJIOg-r3yQ9vc&e=
_______________________________________________
nginx mailing list
nginx@nginx.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.nginx.org_mailman_listinfo_nginx&d=DwIGaQ&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=ZCQk_xaz3T6VBfr2yeoj1xp7l71S7cxV2zGx5Oj3yjM&s=fYHX-hmkaS0qF_i5RTcRpKfzALAMEa6pYb_8U_SGyz0&e=
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
