Re: Using the mirror module

2018-08-14 Thread jlangr
thanks Francis for the additional info, very helpful!

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?2,279036,280872#msg-280872

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx


Re: Using the mirror module

2018-08-14 Thread Francis Daly
On Thu, Aug 09, 2018 at 07:04:50PM -0400, jlangr wrote:

Hi there,

> We got this working to wrap up the day.

Good that you have something working for you.

> My ignorance of nginx meant that I
> was interpreting the example literally--I viewed the "/mirror" as the name
> for the internal route, but that actually becomes part of the URL the mirror
> location proxy forwards to.

It's actually a bit more subtle than that.

The argument to the "mirror" directive is the local url of the subrequest;
it should start with "/" or "@".

Then, in the matching location{}, if you choose to proxy_pass, then the
proxy_pass rules come in to play. And it is proxy_pass that decides what
request to make to the upstream server.

http://nginx.org/r/proxy_pass

> mirror /;

That means that the subrequest will be exactly "/"...

> location = / {

...which will be handled in this location...

> set $upstream_endpoint http://1.2.3.4:3002;
> proxy_pass $upstream_endpoint$request_uri;

...and the proxy_pass argument includes something after the host:port,
so that is what the request to upstream will be.

An alternative could have been to use a named location -- "mirror
@mirror;", with "location @mirror { proxy_pass http://1.2.3.4:3002; }"
-- which would avoid the need for some of the extra variables.


But, once you have something that works, there is no need to change it :-)

Cheers,

f
-- 
Francis Dalyfran...@daoine.org
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx


Re: Using the mirror module

2018-08-09 Thread jlangr
Hi Francis--

We got this working to wrap up the day. My ignorance of nginx meant that I
was interpreting the example literally--I viewed the "/mirror" as the name
for the internal route, but that actually becomes part of the URL the mirror
location proxy forwards to.

http {
server {
listen ;
server_name localhost;

location / {
mirror /;
resolver 8.8.8.8;   # I shouldn't need this, will test tomorrow
proxy_pass http://1.2.3.4:3001;
}

location = / {
internal;
resolver 8.8.8.8;
set $upstream_endpoint http://1.2.3.4:3002;
proxy_pass $upstream_endpoint$request_uri;
}
}
}

That works like a champ. Initially I was trying to use localhost as the
proxy_pass destination, and had turned on port forwarding at my router, but
for whatever reason that was having troubles.

Many thanks in any case!

Jeff

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?2,279036,280832#msg-280832

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx


Re: Using the mirror module

2018-08-09 Thread jlangr
ps how do you format code in these posts? (It's not markdown)

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?2,279036,280833#msg-280833

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx


Re: Using the mirror module

2018-08-09 Thread Francis Daly
On Thu, Aug 09, 2018 at 05:08:25PM -0400, jlangr wrote:

Hi there,

> hmm looks like the mirror kills the url. I need to understand how to do URL
> rewrites, probably, but the original request's path isn't coming across to
> the mirror location.

Does it work for you if you use the Example Configuration in the
documentation?

http://nginx.org/en/docs/http/ngx_http_mirror_module.html

f
-- 
Francis Dalyfran...@daoine.org
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx


Re: Using the mirror module

2018-08-09 Thread jlangr
hmm looks like the mirror kills the url. I need to understand how to do URL
rewrites, probably, but the original request's path isn't coming across to
the mirror location.

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?2,279036,280829#msg-280829

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx


Re: Using the mirror module

2018-08-09 Thread jlangr
Thanks for the posts Kenny & pbooth.

I have this problem as well. I am running nginx 1.15 under MacOS and also
experience this under Windows.

Requests successfully go to the server specified under the location but are
not received at the proxy_pass defined in the mirror location's proxy_pass.
Switching between the two servers (swapping the mirror for the proxy_pass in
the primary location) exhibits the same problem.

My config is:
```
http {
server {
listen ;
server_name  localhost;

location / {
mirror /other;
proxy_pass http://localhost:3001;
}

location = /other {
internal;
proxy_pass http://localhost:3002;
}
}
}

events {
}
```
Based on your other post, it does not look like I need a resolver.

I've also tried with two servers--one localhost and one established server
on another machine.

Any help greatly appreciated!

Jeff

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?2,279036,280828#msg-280828

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx


Re: Using the mirror module

2018-03-14 Thread Roman Arutyunyan
On Wed, Mar 14, 2018 at 08:55:39AM -0300, Kenny Meyer wrote:
> How do you define a resolver?

Use the "resolver" directive:

http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver

This only makes sense if your nginx resolves domain names in runtime.
In your configuration "staging.example.com" is resolved in runtime because
the entire URL contains a variable ($request_uri).  However, if you put an ip
address instead, you will not need a resolver.

> > On 13 Mar, 2018, at 19:36, Roman Arutyunyan  wrote:
> > 
> > On Tue, Mar 13, 2018 at 06:58:25PM -0300, Kenny Meyer wrote:
> >> Hi Roman,
> >> 
> >>> Are there any errors in error.log?
> >> No errors…
> >> 
> >>> And what happens if you switch www.example.com and staging.example.com?
> >> Then I get redirected to staging.example.com and I don’t see any requests 
> >> being logged on example.com
> > 
> > Do you have a resolver defined in the http{} block?
> > Proxying to "http://staging.example.com$request_uri; requires a resolver.
> > Normally, you would have "no resolver defined to resolve ..."
> > error logged if resolver is missing, but you may not see it if, for example,
> > your log level is too high.
> > 
> >>> On 13 Mar, 2018, at 18:34, Roman Arutyunyan  wrote:
> >>> 
> >>> Hi Kenny,
> >>> 
> >>> On Tue, Mar 13, 2018 at 05:37:52PM -0300, Kenny Meyer wrote:
>  Hi,
>  
>  I’m having trouble using the new mirror module. I want to mirror 
>  incoming requests from Nginx to other upstream servers. 1) a production 
>  server 2) a staging server
>  
>  This is my config:
>  
>  server {
>    listen 80 default_server;
>    listen [::]:80 default_server;
>  
>    location / {
>    mirror /mirror;
>    proxy_pass http://www.example.com;
>    }
>  
>    location /mirror {
>    internal;
>    proxy_pass http://staging.example.com$request_uri;
>    }
>  }
>  
>  So, I request http://myserver.com (where Nginx is hosted) and it 
>  successfully redirects me to www.example.com, however I don’t see any 
>  requests hitting staging.example.com. 
>  
>  What could be the error?
> >>> 
> >>> The configuration looks fine.
> >>> Are there any errors in error.log?
> >>> And what happens if you switch www.example.com and staging.example.com?
> >>> 
> >>> -- 
> >>> Roman Arutyunyan
> >>> ___
> >>> nginx mailing list
> >>> nginx@nginx.org
> >>> http://mailman.nginx.org/mailman/listinfo/nginx
> >> 
> >> Kenny Meyer
> >> www.kennymeyer.net
> >> 
> >> ___
> >> nginx mailing list
> >> nginx@nginx.org
> >> http://mailman.nginx.org/mailman/listinfo/nginx
> > 
> > -- 
> > Roman Arutyunyan
> > ___
> > nginx mailing list
> > nginx@nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx
> 
> Kenny Meyer
> www.kennymeyer.net
> 
> ___
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-- 
Roman Arutyunyan
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Using the mirror module

2018-03-14 Thread Peter Booth
Suggestion:

Define two more locations - one that proxies www.example.com and another that 
proxies staging.example.com. If both locations work then your problem is 
probably mirroring. If one doesn’t work then the issue is your configuration 
and not mirroring. Either way you have reduced the size of your problem space.

Peter

Sent from my iPhone

> On Mar 13, 2018, at 5:58 PM, Kenny Meyer  wrote:
> 
> Hi Roman,
> 
>> Are there any errors in error.log?
> No errors…
> 
>> And what happens if you switch www.example.com and staging.example.com?
> Then I get redirected to staging.example.com and I don’t see any requests 
> being logged on example.com
> 
> 
> 
>> On 13 Mar, 2018, at 18:34, Roman Arutyunyan  wrote:
>> 
>> Hi Kenny,
>> 
>>> On Tue, Mar 13, 2018 at 05:37:52PM -0300, Kenny Meyer wrote:
>>> Hi,
>>> 
>>> I’m having trouble using the new mirror module. I want to mirror incoming 
>>> requests from Nginx to other upstream servers. 1) a production server 2) a 
>>> staging server
>>> 
>>> This is my config:
>>> 
>>> server {
>>>   listen 80 default_server;
>>>   listen [::]:80 default_server;
>>> 
>>>   location / {
>>>   mirror /mirror;
>>>   proxy_pass http://www.example.com;
>>>   }
>>> 
>>>   location /mirror {
>>>   internal;
>>>   proxy_pass http://staging.example.com$request_uri;
>>>   }
>>> }
>>> 
>>> So, I request http://myserver.com (where Nginx is hosted) and it 
>>> successfully redirects me to www.example.com, however I don’t see any 
>>> requests hitting staging.example.com. 
>>> 
>>> What could be the error?
>> 
>> The configuration looks fine.
>> Are there any errors in error.log?
>> And what happens if you switch www.example.com and staging.example.com?
>> 
>> -- 
>> Roman Arutyunyan
>> ___
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
> 
> Kenny Meyer
> www.kennymeyer.net
> 
> ___
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Using the mirror module

2018-03-14 Thread Kenny Meyer
How do you define a resolver?

> On 13 Mar, 2018, at 19:36, Roman Arutyunyan  wrote:
> 
> On Tue, Mar 13, 2018 at 06:58:25PM -0300, Kenny Meyer wrote:
>> Hi Roman,
>> 
>>> Are there any errors in error.log?
>> No errors…
>> 
>>> And what happens if you switch www.example.com and staging.example.com?
>> Then I get redirected to staging.example.com and I don’t see any requests 
>> being logged on example.com
> 
> Do you have a resolver defined in the http{} block?
> Proxying to "http://staging.example.com$request_uri; requires a resolver.
> Normally, you would have "no resolver defined to resolve ..."
> error logged if resolver is missing, but you may not see it if, for example,
> your log level is too high.
> 
>>> On 13 Mar, 2018, at 18:34, Roman Arutyunyan  wrote:
>>> 
>>> Hi Kenny,
>>> 
>>> On Tue, Mar 13, 2018 at 05:37:52PM -0300, Kenny Meyer wrote:
 Hi,
 
 I’m having trouble using the new mirror module. I want to mirror incoming 
 requests from Nginx to other upstream servers. 1) a production server 2) a 
 staging server
 
 This is my config:
 
 server {
   listen 80 default_server;
   listen [::]:80 default_server;
 
   location / {
   mirror /mirror;
   proxy_pass http://www.example.com;
   }
 
   location /mirror {
   internal;
   proxy_pass http://staging.example.com$request_uri;
   }
 }
 
 So, I request http://myserver.com (where Nginx is hosted) and it 
 successfully redirects me to www.example.com, however I don’t see any 
 requests hitting staging.example.com. 
 
 What could be the error?
>>> 
>>> The configuration looks fine.
>>> Are there any errors in error.log?
>>> And what happens if you switch www.example.com and staging.example.com?
>>> 
>>> -- 
>>> Roman Arutyunyan
>>> ___
>>> nginx mailing list
>>> nginx@nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>> 
>> Kenny Meyer
>> www.kennymeyer.net
>> 
>> ___
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
> 
> -- 
> Roman Arutyunyan
> ___
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

Kenny Meyer
www.kennymeyer.net

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Using the mirror module

2018-03-13 Thread Roman Arutyunyan
On Tue, Mar 13, 2018 at 06:58:25PM -0300, Kenny Meyer wrote:
> Hi Roman,
> 
> > Are there any errors in error.log?
> No errors…
> 
> > And what happens if you switch www.example.com and staging.example.com?
> Then I get redirected to staging.example.com and I don’t see any requests 
> being logged on example.com

Do you have a resolver defined in the http{} block?
Proxying to "http://staging.example.com$request_uri; requires a resolver.
Normally, you would have "no resolver defined to resolve ..."
error logged if resolver is missing, but you may not see it if, for example,
your log level is too high.

> > On 13 Mar, 2018, at 18:34, Roman Arutyunyan  wrote:
> > 
> > Hi Kenny,
> > 
> > On Tue, Mar 13, 2018 at 05:37:52PM -0300, Kenny Meyer wrote:
> >> Hi,
> >> 
> >> I’m having trouble using the new mirror module. I want to mirror incoming 
> >> requests from Nginx to other upstream servers. 1) a production server 2) a 
> >> staging server
> >> 
> >> This is my config:
> >> 
> >> server {
> >>listen 80 default_server;
> >>listen [::]:80 default_server;
> >> 
> >>location / {
> >>mirror /mirror;
> >>proxy_pass http://www.example.com;
> >>}
> >> 
> >>location /mirror {
> >>internal;
> >>proxy_pass http://staging.example.com$request_uri;
> >>}
> >> }
> >> 
> >> So, I request http://myserver.com (where Nginx is hosted) and it 
> >> successfully redirects me to www.example.com, however I don’t see any 
> >> requests hitting staging.example.com. 
> >> 
> >> What could be the error?
> > 
> > The configuration looks fine.
> > Are there any errors in error.log?
> > And what happens if you switch www.example.com and staging.example.com?
> > 
> > -- 
> > Roman Arutyunyan
> > ___
> > nginx mailing list
> > nginx@nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx
> 
> Kenny Meyer
> www.kennymeyer.net
> 
> ___
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-- 
Roman Arutyunyan
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Using the mirror module

2018-03-13 Thread Peter Booth
This is the point where I would jump to using the debug log. You need to build 
you nginx binary 
with —with-debug switch and change the log level to debug innginx.conf. Debug 
generates a *huge* 
amount of logs but it really is invaluable.

I would also want to double check what is actually happening and use ss or 
tcpdump 
to confirm that no request is sent to your staging destination.

 I’m assuming that both ww.example.com and staging.example.com are hosted 
on different hosts, different IPs and are both functional.

Peter

 

> On Mar 13, 2018, at 5:58 PM, Kenny Meyer  wrote:
> 
> Hi Roman,
> 
>> Are there any errors in error.log?
> No errors…
> 
>> And what happens if you switch www.example.com and staging.example.com?
> Then I get redirected to staging.example.com and I don’t see any requests 
> being logged on example.com
> 
> 
> 
>> On 13 Mar, 2018, at 18:34, Roman Arutyunyan  wrote:
>> 
>> Hi Kenny,
>> 
>> On Tue, Mar 13, 2018 at 05:37:52PM -0300, Kenny Meyer wrote:
>>> Hi,
>>> 
>>> I’m having trouble using the new mirror module. I want to mirror incoming 
>>> requests from Nginx to other upstream servers. 1) a production server 2) a 
>>> staging server
>>> 
>>> This is my config:
>>> 
>>> server {
>>>   listen 80 default_server;
>>>   listen [::]:80 default_server;
>>> 
>>>   location / {
>>>   mirror /mirror;
>>>   proxy_pass http://www.example.com;
>>>   }
>>> 
>>>   location /mirror {
>>>   internal;
>>>   proxy_pass http://staging.example.com$request_uri;
>>>   }
>>> }
>>> 
>>> So, I request http://myserver.com (where Nginx is hosted) and it 
>>> successfully redirects me to www.example.com, however I don’t see any 
>>> requests hitting staging.example.com. 
>>> 
>>> What could be the error?
>> 
>> The configuration looks fine.
>> Are there any errors in error.log?
>> And what happens if you switch www.example.com and staging.example.com?
>> 
>> -- 
>> Roman Arutyunyan
>> ___
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
> 
> Kenny Meyer
> www.kennymeyer.net
> 
> ___
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Using the mirror module

2018-03-13 Thread Kenny Meyer
Hi Roman,

> Are there any errors in error.log?
No errors…

> And what happens if you switch www.example.com and staging.example.com?
Then I get redirected to staging.example.com and I don’t see any requests being 
logged on example.com



> On 13 Mar, 2018, at 18:34, Roman Arutyunyan  wrote:
> 
> Hi Kenny,
> 
> On Tue, Mar 13, 2018 at 05:37:52PM -0300, Kenny Meyer wrote:
>> Hi,
>> 
>> I’m having trouble using the new mirror module. I want to mirror incoming 
>> requests from Nginx to other upstream servers. 1) a production server 2) a 
>> staging server
>> 
>> This is my config:
>> 
>> server {
>>listen 80 default_server;
>>listen [::]:80 default_server;
>> 
>>location / {
>>mirror /mirror;
>>proxy_pass http://www.example.com;
>>}
>> 
>>location /mirror {
>>internal;
>>proxy_pass http://staging.example.com$request_uri;
>>}
>> }
>> 
>> So, I request http://myserver.com (where Nginx is hosted) and it 
>> successfully redirects me to www.example.com, however I don’t see any 
>> requests hitting staging.example.com. 
>> 
>> What could be the error?
> 
> The configuration looks fine.
> Are there any errors in error.log?
> And what happens if you switch www.example.com and staging.example.com?
> 
> -- 
> Roman Arutyunyan
> ___
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

Kenny Meyer
www.kennymeyer.net

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Using the mirror module

2018-03-13 Thread Roman Arutyunyan
Hi Kenny,

On Tue, Mar 13, 2018 at 05:37:52PM -0300, Kenny Meyer wrote:
> Hi,
> 
> I’m having trouble using the new mirror module. I want to mirror incoming 
> requests from Nginx to other upstream servers. 1) a production server 2) a 
> staging server
> 
> This is my config:
> 
> server {
> listen 80 default_server;
> listen [::]:80 default_server;
> 
> location / {
> mirror /mirror;
> proxy_pass http://www.example.com;
> }
> 
> location /mirror {
> internal;
> proxy_pass http://staging.example.com$request_uri;
> }
> }
> 
> So, I request http://myserver.com (where Nginx is hosted) and it successfully 
> redirects me to www.example.com, however I don’t see any requests hitting 
> staging.example.com. 
> 
> What could be the error?

The configuration looks fine.
Are there any errors in error.log?
And what happens if you switch www.example.com and staging.example.com?

-- 
Roman Arutyunyan
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx