Re: 403 forbidden with lynx www browser

[B.R.]

​Hello,​

On Mon, Nov 23, 2015 at 12:23 AM, Sylvain BERTRAND <
sylvain.bertr...@gmail.com> wrote:

> That's why I'm posting here: *Only nginx* www sites does block lynx.
> Something
> is not right there: a default aggressive blocking policy from nginx?
>

​There is a difference between 'only websites I visit which happen to use
nginx' and 'every nginx websites'
Stock nginx does nothing else but serving data to clients in the most
simple way and RFC-compliant.​

As Francis pointed out, it is most probably a deliberate configuration from
those websites you visit which may consider your user-agent string looking
too much like crawlers/bots.
As Francis also told you, if nginx.org or nginx.com websites are accessible
to you without harm, then the nginx product is not the source of your
trouble, and you might be willing to contact the adminitrators of the
websites you visit to complain and talk about a solution.
You might also change your user-agent string to look like more 'official'
ones, unless you wish to defend the use of lynx, which might be a
meaningful quest.

Best of luck,
---
*B. R.*
​
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: 403 forbidden with lynx www browser

[Francis Daly]

On Mon, Nov 23, 2015 at 10:23:20AM +1100, Sylvain BERTRAND wrote:
> On Sun, Nov 22, 2015 at 05:25:11PM +, Francis Daly wrote:

Hi there,

> > Probably they have configured their servers to deny any request that
> > includes "libwww" in the User-Agent.
> 
> I'm about to block sending the user agent for good and all www sites.
> (or I will send a custom one).

Possibly something like "lynx, but with a non-default user agent string"
will get their attention. (Probably not, though.)

> > But unless the nginx.org (or possibly nginx.com) web sites are among
> > the sites that block you, there's probably not much that readers of this
> > list can do about it.
> 
> That's why I'm posting here: *Only nginx* www sites does block lynx. Something
> is not right there: a default aggressive blocking policy from nginx?

There is no default policy in stock nginx, or in any distributed nginx
that I'm aware of.

My suspicion is that these sites that run nginx also run some back-end
framework, and that that framework may have some suggested configuration
for nginx which blocks access based on user-agent matching.

If there is any hint in the response headers what the full server-side
suite is, that may be a good place to start looking for a more general
future solution.

Cheers,

f
-- 
Francis Dalyfran...@daoine.org

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Complex url rewriting

[B.R.]

Hello,

You do not necessarily need to *redirect* *per se*, but you wish content to
be served by your index.php files.

Would
location /index.php {
location ~* (?:.*/index.php)(.*) {
fastcgi_param SCRIPT_FILENAME $document_root$1;
fastcgi_pass ;
}
}
do the job? (untested)
---
*B. R.*

On Mon, Nov 23, 2015 at 7:19 PM, lmauldinpe15  wrote:

> Another note, on some of the application sub directories, I need to emulate
> this rule from .htaccess: RewriteRule ^(.*)$ %{DOCUMENT_ROOT}/index.php/$1
> [QSA,L]
>
> Posted at Nginx Forum:
> https://forum.nginx.org/read.php?2,263024,263026#msg-263026
>
> ___
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: 403 forbidden with lynx www browser

[B.R.]

You missed the parts where Francis and I suggested about testing against
nginx.org and/or nginx.com websites.
Is those reply correctly, the nginx product is definitely not the source of
the trouble.
I could also provide you with domains I serve with nginx, which do not use
any sort of user-agent filtering, through private channels, if necessary.

​Stuff like common rulesets or IDS tend to 'auto-configure' products with
rules (which then might be shared) to the problem would still not come from
nginx itself.​
---
*B. R.*

On Mon, Nov 23, 2015 at 8:31 PM, Nelson, Erik - 2 <
erik.l.nel...@bankofamerica.com> wrote:

> B.R. Monday, November 23, 2015 2:26 PM
> On Mon, Nov 23, 2015 at 12:23 AM, Sylvain BERTRAND <
> sylvain.bertr...@gmail.com> wrote:
> >>That's why I'm posting here: *Only nginx* www sites does block lynx.
> Something
> >>is not right there: a default aggressive blocking policy from nginx?
>
> >​There is a difference between 'only websites I visit which happen to use
> nginx' and 'every nginx websites'
>
> That may be true, but it's missing the point.  This has come up before on
> the ML.  The point is that he has observed something systemic.  Maybe it's
> a default configuration that's tighter (as he suggested), maybe it's that
> admins who use nginx just hate lynx, or maybe it's something else.
>
> Whatever it is, there's *something* about nginx that's different.
>
> --
> This message, and any attachments, is for the intended recipient(s) only,
> may contain information that is privileged, confidential and/or proprietary
> and subject to important terms and conditions available at
> http://www.bankofamerica.com/emaildisclaimer.   If you are not the
> intended recipient, please delete this message.
> ___
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

[nginx] Configure: fixed using OpenSSL include paths.

[Maxim Dounin]

details:   http://hg.nginx.org/nginx/rev/520ec1917f1d
branches:  
changeset: 6304:520ec1917f1d
user:  Maxim Dounin 
date:  Mon Nov 23 22:48:31 2015 +0300
description:
Configure: fixed using OpenSSL include paths.

diffstat:

 auto/lib/openssl/conf |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (11 lines):

diff --git a/auto/lib/openssl/conf b/auto/lib/openssl/conf
--- a/auto/lib/openssl/conf
+++ b/auto/lib/openssl/conf
@@ -105,6 +105,7 @@ else
 
 if [ $ngx_found = yes ]; then
 have=NGX_SSL . auto/have
+CORE_INCS="$CORE_INCS $ngx_feature_path"
 CORE_LIBS="$CORE_LIBS $ngx_feature_libs $NGX_LIBDL"
 OPENSSL=YES
 fi

___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

RE: 403 forbidden with lynx www browser

[Nelson, Erik - 2]

B.R. Monday, November 23, 2015 2:26 PM
On Mon, Nov 23, 2015 at 12:23 AM, Sylvain BERTRAND  
wrote:
>>That's why I'm posting here: *Only nginx* www sites does block lynx. Something
>>is not right there: a default aggressive blocking policy from nginx?

>​There is a difference between 'only websites I visit which happen to use 
>nginx' and 'every nginx websites'

That may be true, but it's missing the point.  This has come up before on the 
ML.  The point is that he has observed something systemic.  Maybe it's a 
default configuration that's tighter (as he suggested), maybe it's that admins 
who use nginx just hate lynx, or maybe it's something else.

Whatever it is, there's *something* about nginx that's different.

--
This message, and any attachments, is for the intended recipient(s) only, may 
contain information that is privileged, confidential and/or proprietary and 
subject to important terms and conditions available at 
http://www.bankofamerica.com/emaildisclaimer.   If you are not the intended 
recipient, please delete this message.
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: 400 Error on % !!

[B.R.]

The '%' character has a meaning in the HTTP URI grammar, indicating that
the following bytes are hexadecimal value representing a unicode character
(see https://tools.ietf.org/html/rfc3986#section-2.1).
If you try to implement what you suggest, you will basically corrupt URIs
from someone tring to access your website with Unicode-encoded strings,
which might be perfectly valid.

​The only solution I find viable is that you remove percent character from
your URIs which are not representing Unicode characters, for example by
encoding '%'.​ The percent-encoded version of '%' is '%25'.
---
*B. R.*

On Mon, Nov 23, 2015 at 9:40 AM, shahzaib shahzaib 
wrote:

> Hi,
>
>   Thanks for the reply, now we've tons of these URLs which are not
> properly encoded. Can we redirect '%' request to the same URL by excluding
> '%' ? Such as
>
> http://domain.com/video/100%-working
>
> to
>
> http://domain.com/video/100-working
>
> Regards.
> Shahzaib
>
> On Mon, Nov 23, 2015 at 1:20 PM, nanaya  wrote:
>
>>
>>
>> On Mon, Nov 23, 2015, at 05:17 PM, shahzaib shahzaib wrote:
>> > Hi,
>> >
>> >We've encountered with 400 Bad request error on nginx reverse proxy
>> in
>> > front of apache. Here is the attached link :
>> >
>> > http://prntscr.com/95wlsl
>> >
>> > If we remove '%' from the URL, it works fine. What could be the issue ?
>> >
>>
>> `%-` isn't a valid percent-encoding (and thus a bad request). Try
>> encoding the url properly.
>>
>> ___
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>>
>
>
> ___
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Complex url rewriting

[lmauldinpe15]

Another note, on some of the application sub directories, I need to emulate
this rule from .htaccess: RewriteRule ^(.*)$ %{DOCUMENT_ROOT}/index.php/$1
[QSA,L]

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?2,263024,263026#msg-263026

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: 400 Error on % !!

[Francis Daly]

On Mon, Nov 23, 2015 at 08:34:41PM +0100, B.R. wrote:
> On Mon, Nov 23, 2015 at 9:40 AM, shahzaib shahzaib 
> wrote:

Hi there,

[mostly addressed to the original poster]

> >   Thanks for the reply, now we've tons of these URLs which are not
> > properly encoded.

> The only solution I find viable is that you remove percent character from
> your URIs which are not representing Unicode characters, for example by
> encoding '%'. The percent-encoded version of '%' is '%25'.

The right answer is to fix those tons of urls at the place they are written.

If whoever wrote them had written "domain.org" instead of "domain.com",
you'd have to fix them (or take control of domain.org).

This is broadly similar to that.

Either strip the % in the cases where you know it was written unencoded;
or encode it to %25 in the same cases.

> > Can we redirect '%' request to the same URL by excluding
> > '%' ? Such as
> >
> > http://domain.com/video/100%-working
> >
> > to
> >
> > http://domain.com/video/100-working

In theory, you could have a front-end web service which accepted all
requests, and for the specific ones that are clearly broken like this,
could redirect to the fixed version; otherwise is could pass the request
through to the back-end (current) web server. (Or it could pass through
the fixed version; but that feels like it would be even more complicated.)

It could only work if you know the broken urls, though --
the url /video/100-working is exactly equivalent to the url
/v%69deo/100%2dworkin%67, so you would not want to %-strip that one. And
if you have one request for /video/50%good and one for /video/50%bad
-- is the second one encoded or not? % is followed by two hexadecimal
characters, which should mean "it is correctly encoded".

If you wanted nginx to be this front-end web service, I think that you
would need code-level changes in your version to get it to accept the
broken input. It is not "just" configuration that would achieve it.

Good luck with it,

f
-- 
Francis Dalyfran...@daoine.org

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Complex url rewriting

[lmauldinpe15]

Did you mean to use nested location blocks?  I tried it but it didn't work. 
Here is the relevant part of my configuration file:

 # define web root
root   /var/www/html/public;
index  index.php index.html;

location /index.php {
location ~* (?:.*/index.php)(.*) {
fastcgi_param SCRIPT_FILENAME $document_root$1;
fastcgi_pass   unix:/var/run/php5-fpm.sock;
includefastcgi_params;
}
}

location / {
try_files $uri $uri/ =404;
}
location ~ \.php$ {
fastcgi_pass   unix:/var/run/php5-fpm.sock;
try_files $uri =404;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
includefastcgi_params;
}

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?2,263024,263035#msg-263035

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

400 Error on % !!

[shahzaib shahzaib]

Hi,

   We've encountered with 400 Bad request error on nginx reverse proxy in
front of apache. Here is the attached link :

http://prntscr.com/95wlsl

If we remove '%' from the URL, it works fine. What could be the issue ?

Regards.
Shahzaib


Need to send me private email? I use Virtru
.
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: 400 Error on % !!

[nanaya]

On Mon, Nov 23, 2015, at 05:17 PM, shahzaib shahzaib wrote:
> Hi,
> 
>We've encountered with 400 Bad request error on nginx reverse proxy in
> front of apache. Here is the attached link :
> 
> http://prntscr.com/95wlsl
> 
> If we remove '%' from the URL, it works fine. What could be the issue ?
> 

`%-` isn't a valid percent-encoding (and thus a bad request). Try
encoding the url properly.

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: 400 Error on % !!

[shahzaib shahzaib]

Hi,

  Thanks for the reply, now we've tons of these URLs which are not properly
encoded. Can we redirect '%' request to the same URL by excluding '%' ?
Such as

http://domain.com/video/100%-working

to

http://domain.com/video/100-working

Regards.
Shahzaib

On Mon, Nov 23, 2015 at 1:20 PM, nanaya  wrote:

>
>
> On Mon, Nov 23, 2015, at 05:17 PM, shahzaib shahzaib wrote:
> > Hi,
> >
> >We've encountered with 400 Bad request error on nginx reverse proxy in
> > front of apache. Here is the attached link :
> >
> > http://prntscr.com/95wlsl
> >
> > If we remove '%' from the URL, it works fine. What could be the issue ?
> >
>
> `%-` isn't a valid percent-encoding (and thus a bad request). Try
> encoding the url properly.
>
> ___
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

[ANN] OpenResty 1.9.3.2 released

[Yichun Zhang (agentzh)]

Hi guys,

I am glad to announce the new formal release, 1.9.3.2, of the
OpenResty bundle:

https://openresty.org/#Download

The first highlight of this release is the new *_by_lua_block {} directives
added in the ngx_http_lua module.

For example, instead of writing

content_by_lua '
ngx.say("Hello, OpenResty\'s world!\\n")
';

We can just write

content_by_lua_block {
ngx.say("Hello, OpenResty's world!\n")
}

No ugly special character escaping is needed in the Lua source any
more in the latter form :)

The second highlight of this release is the new (experimental) support for
Windows using the MinGW gcc toolchain. You can download the pre-built Win32
binaries from this zip package:

https://openresty.org/download/ngx_openresty-1.9.3.2-win32.zip

For detailed usage on Windows, please check out the following document:

https://github.com/openresty/ngx_openresty/blob/master/doc/README-win32.md

If you want to build on Windows yourself, then this document contains
detailed instructions as well.

We will release a corresponding Win32 binary package for every release
of the OpenResty source package from now on.

The Windows build is still experimental and should be used for development only.

We have plans to migrate to the Microsoft Visual Studio compiler
toolchain and to resolve existing limitations on Windows in the near
future.

Changes since the last (formal) release, 1.9.3.1:

 *   feature: added support for compiling on Windows using the MinGW
 gcc toolchain to the build system. See the document for more
 details:

 https://github.com/openresty/ngx_openresty/blob/master/doc/README-win32.md

 *   upgraded the ngx_lua module to 0.9.19.

 *   feature: implemented "*_by_lua_block {}" directives for all
 the existing *_by_lua directives so that we no longer have
 to escape special characters while inlining Lua source
 inside the "nginx.conf" file.

 *   feature: now we support LuaJIT 2 on Windows (in the form of
 "lua51.dll").

 *   feature: initial fixes when being used with the new
 "ngx_http_v2" module since nginx 1.9.5. thanks itpp16 for
 the patches.

 *   bugfix: fixed errors and warnings with C compilers without
 variadic macro support.

 *   bugfix: subrequest response status codes between the range
 100 .. 299 (inclusive) might get lost in the return values
 of ngx.location.capture*() calls. thanks Igor Clark for the
 report.

 *   bugfix: we might return the wrong shm zone in the public C
 API function "ngx_http_lua_find_zone()". thanks qlee001 for
 the report.

 *   bugfix: the user specified "./configure"'s "--with-cc-opt"
 and "--with-ld-opt" might override the
 "LUAJIT_INC"/"LUAJIT_LIB" and "LUA_INC"/"LUA_LIB"
 environment settings. thanks Julian Gonggrijp for the
 report.

 *   bugfix: setting builtin request header "Upgrade" via
 ngx.req.set_header and etc might not take effect with some
 builtin nginx modules.

 *   bugfix: setting builtin request headers "Depth",
 "Destination", "Overwrite", and "Date" via
 ngx.req.set_header() and etc might not take effect at least
 with ngx_http_dav_module. thanks Igor Clark for the report.

 *   bugfix: fixed typos due to copy mistakes in some error
 messages.

 *   bugfix: fixed one "-Wmaybe-uninitialized" warning when
 compiling with "gcc -Os".

 *   bugfix: use of shared dicts resulted in (unwanted)
 registrations of shared dict metatables on *all* the
 lightuserdata in the Lua space. thanks helloyi for the
 report and patch.

 *   bugfix: if a 3rd-party module calls
 "ngx_http_conf_get_module_srv_conf" to fetch its current
 "srv_conf" construct in its "merge_srv_conf" callback, then
 use of init_worker_by_lua might lead to segmentation faults
 (the same also applied to merge_loc_conf). thanks chiyouhen
 for the report and patch.

 *   bugfix: the "if_unmodified_since" "shortcut" field in
 "ngx_http_headers_in_t" was first added in nginx 0.9.2.

 *   bugfix: ngx.req.clear_header/ngx.req.set_header: we did not
 update the shortcut fields in "ngx_http_headers_in_t" added
 since nginx 1.3.3 which may confuse other nginx modules
 accessing them.

 *   bugfix: setting "Content-Type" response values including ";
 charset=xxx" via the ngx.header API might bypass the MIME
 type checks in other nginx modules like ngx_gzip. thanks
 Andreas Fischer for the report.

 *   bugfix: typo fixes in some debug logging messages. thanks
 doujiang for the patch.

 *   optimize: fixed the hash-table initial sizes of the cosocket
 metatables. thanks ops-dev-cn for the patch.

 *   tests: removed the useless "use lib" directives 

Re: Unit testing approach for nginx modules

[Anatoli Marinov]

Hi Maxim,
How these tests could be run? Do I need something special installed?

On Mon, Nov 23, 2015 at 2:42 AM, Maxim Dounin  wrote:

> Hello!
>
> On Sat, Nov 21, 2015 at 09:40:57PM +, Ritesh Jha wrote:
>
> > Hello everyone,
> > We are developing nginx modules to implement few usecases in our
> > product. Most of the other usecases cases have been implemented
> > using Java. At my office we follow TDD for Java development. TDD
> > for Java development is easy due to availability of unit-testing
> > and mocking frameworks. We are wondering if we can follow TDD
> > for development of nginx modules as well. We have tried couple
> > of unit-testing frameworks for C (Unity and CMocka) but we have
> > found it very difficult to write useful testcases using these
> > frameworks.
> >
> > Can you please suggest a suitable approach? Also if unit testing
> > is not the way to go, then what should be the approach for
> > developing, testing and maintaining large nginx modules?
>
> We have a test suite we use in nginx development, available here:
>
> http://hg.nginx.org/nginx-tests
>
> It was originally written by me for testing both nginx itself and
> my modules for nginx.  And example use in a module can be seen
> here:
>
> http://mdounin.ru/hg/ngx_http_bytes_filter_module/file/57365655ee44/t
>
> --
> Maxim Dounin
> http://nginx.org/
>
> ___
> nginx-devel mailing list
> nginx-devel@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

Re: 403 forbidden with lynx www browser

[Sylvain BERTRAND]

On Mon, Nov 23, 2015 at 07:31:15PM +, Nelson, Erik - 2 wrote:
> B.R. Monday, November 23, 2015 2:26 PM
> On Mon, Nov 23, 2015 at 12:23 AM, Sylvain BERTRAND 
>  wrote:
> >>That's why I'm posting here: *Only nginx* www sites does block lynx. 
> >>Something
> >>is not right there: a default aggressive blocking policy from nginx?
> 
> >​There is a difference between 'only websites I visit which happen to
> >use nginx' and 'every nginx websites'
> 
> That may be true, but it's missing the point.  This has come up before on the
> ML.  The point is that he has observed something systemic.  Maybe it's a
> default configuration that's tighter (as he suggested), maybe it's that
> admins who use nginx just hate lynx, or maybe it's something else.
> 
> Whatever it is, there's *something* about nginx that's different.

You got it right.

The last web site... a friend sent me the link (:p), had to remove the 
user-agent from http headers.
http://www.rawstory.com/2015/11/hacker-collective-anonymous-claims-isis-has-plans-for-more-attacks-on-sunday/

-- 
Sylvain

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: 403 forbidden with lynx www browser

[B.R.]

*There is none so deaf than those who will not hear.* :o|
​Well​ if you two understand each other, find where nginx handles some
user-agents differently than others. I am sure the developers would be more
than glad to learn about it. Everyone is, actually.
---
*B. R.*

On Tue, Nov 24, 2015 at 12:00 AM, Sylvain BERTRAND <
sylvain.bertr...@gmail.com> wrote:

> On Mon, Nov 23, 2015 at 07:31:15PM +, Nelson, Erik - 2 wrote:
> > B.R. Monday, November 23, 2015 2:26 PM
> > On Mon, Nov 23, 2015 at 12:23 AM, Sylvain BERTRAND <
> sylvain.bertr...@gmail.com> wrote:
> > >>That's why I'm posting here: *Only nginx* www sites does block lynx.
> Something
> > >>is not right there: a default aggressive blocking policy from nginx?
> >
> > >​There is a difference between 'only websites I visit which happen to
> > >use nginx' and 'every nginx websites'
> >
> > That may be true, but it's missing the point.  This has come up before
> on the
> > ML.  The point is that he has observed something systemic.  Maybe it's a
> > default configuration that's tighter (as he suggested), maybe it's that
> > admins who use nginx just hate lynx, or maybe it's something else.
> >
> > Whatever it is, there's *something* about nginx that's different.
>
> You got it right.
>
> The last web site... a friend sent me the link (:p), had to remove the
> user-agent from http headers.
>
> http://www.rawstory.com/2015/11/hacker-collective-anonymous-claims-isis-has-plans-for-more-attacks-on-sunday/
>
> --
> Sylvain
>
> ___
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: [PATCH] HTTP: implement 'connect' and 'close' phases

[Maxim Dounin]

Hello!

On Mon, Nov 23, 2015 at 03:05:07PM +, Alessandro Ghedini wrote:

> On Fri, Nov 20, 2015 at 05:46:12pm +0300, Maxim Dounin wrote:
> > Hello!
> 
> Hi again,
> 
> > On Fri, Nov 20, 2015 at 09:45:53AM +, Alessandro Ghedini wrote:
> > 
> > > # HG changeset patch
> > > # User Alessandro Ghedini 
> > > # Date 1447956026 0
> > > #  Thu Nov 19 18:00:26 2015 +
> > > # Node ID 9d265c320050a00ff24fa8d84371701e46147e8a
> > > # Parent  bec5b3093337708cbdb59f9fc253f8e1cd6d7848
> > > HTTP: implement 'connect' and 'close' phases
> > > 
> > > This patch adds the NGX_HTTP_CONNECT_PHASE and NGX_HTTP_CLOSE_PHASE
> > > phases.
> > > 
> > > Handlers for these phases are called when a connection is estabilished
> > > and when it is closed, and they take a ngx_connection_t as argument
> > > instead of ngx_http_request_t like the other phase handlers.
> > > 
> > > These can be useful for keeping track of TCP connections for debugging,
> > > monitoring and logging purposes, and can also be used to apply custom
> > > configurations (e.g. socket options).
> > > 
> > > This patch also adds a "ctx" field to ngx_connection_t, to be used by
> > > modules to store their own context structures (just like the ctx field
> > > in ngx_http_request_t).
> > 
> > Phases are request processing phases, and what you are trying to 
> > do doesn't looks like request processing.  Additionally, ctx in 
> > ngx_connection_t implies noticeable memory overhead for keepalive 
> > connections.
> 
> I think I can do without the ctx field. Would removing that help in getting
> this functionality (or something similar) merged?

Unlikely.  Though it's certainly a prerequisite.

> > Instead, consider:
> > 
> > - starting your processing at any request processing stage as 
> >   needed;
> > 
> > - using a connection pool cleanup handler if you want to track 
> >   connection termination;
> 
> The problem with the above is that the pool cleanup callbacks seem to be
> called too late, after the connection has already been closed, so I get an
> invalid socket (basically I need the socket to retrieve some information
> using getsockopt()).

If you want to collect something using getsockopt(), consider 
doing this in the log phase.

Instead of trying to introduce a generic mechanism you may want to 
focus on what you want to do and if it's possible to do this with 
already existing mechanisms.

-- 
Maxim Dounin
http://nginx.org/

___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

Re: [PATCH] HTTP: implement 'connect' and 'close' phases

[Alessandro Ghedini]

On Fri, Nov 20, 2015 at 05:46:12pm +0300, Maxim Dounin wrote:
> Hello!

Hi again,

> On Fri, Nov 20, 2015 at 09:45:53AM +, Alessandro Ghedini wrote:
> 
> > # HG changeset patch
> > # User Alessandro Ghedini 
> > # Date 1447956026 0
> > #  Thu Nov 19 18:00:26 2015 +
> > # Node ID 9d265c320050a00ff24fa8d84371701e46147e8a
> > # Parent  bec5b3093337708cbdb59f9fc253f8e1cd6d7848
> > HTTP: implement 'connect' and 'close' phases
> > 
> > This patch adds the NGX_HTTP_CONNECT_PHASE and NGX_HTTP_CLOSE_PHASE
> > phases.
> > 
> > Handlers for these phases are called when a connection is estabilished
> > and when it is closed, and they take a ngx_connection_t as argument
> > instead of ngx_http_request_t like the other phase handlers.
> > 
> > These can be useful for keeping track of TCP connections for debugging,
> > monitoring and logging purposes, and can also be used to apply custom
> > configurations (e.g. socket options).
> > 
> > This patch also adds a "ctx" field to ngx_connection_t, to be used by
> > modules to store their own context structures (just like the ctx field
> > in ngx_http_request_t).
> 
> Phases are request processing phases, and what you are trying to 
> do doesn't looks like request processing.  Additionally, ctx in 
> ngx_connection_t implies noticeable memory overhead for keepalive 
> connections.

I think I can do without the ctx field. Would removing that help in getting
this functionality (or something similar) merged?

> Instead, consider:
> 
> - starting your processing at any request processing stage as 
>   needed;
> 
> - using a connection pool cleanup handler if you want to track 
>   connection termination;

The problem with the above is that the pool cleanup callbacks seem to be
called too late, after the connection has already been closed, so I get an
invalid socket (basically I need the socket to retrieve some information
using getsockopt()).

Thanks for your help.

Cheers

___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

Re: bug of discarding request body

[Shuxin Yang]

Hi, Maxim:

Thank you very much for the comment, and sorry for my long previous 
email.


I guess you might misunderstand my previous email. Basically what I 
try to say
is that the *OLD* bug (ticket/669 as you mentioned) is seen on the 
*PRISTINE* *NEW*
1.9.7 release. The attached script can reproduce the problem by simply 
invoke

"./a.sh"

   The a.sh donwload the 1.9.7 release, build it *without* any 3rd 
party module.


   The problem is triggered by turning off unbuffered-uploading in 
proxy server.
IIRC, when people report ticket/669, unbuffered-uploading was not 
available.


   I guess we ngx_http_discard_request_body() should return NGX_AGAIN
instead NGX_OK if discarding body is in progress.

   Also see the following interleaving response.

Best Regards
Shuxin


On 11/22/2015 05:24 PM, Maxim Dounin wrote:

There is no problem in returning a response before reading the
whole body.


Ditto!



It looks like you've run into the old bug in the proxy module,
which doesn't handle such responses in keepalive connections
properly, see additional details here:

https://trac.nginx.org/nginx/ticket/669

Correct fix would be to stop nginx from re-using upstream
connections where a request wasn't completely sent.

We cannot tell if it is complete or not if 
ngx_http_discard_request_body() always

returns NGX_OK


repro_400.tar.gz
Description: application/gzip
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Complex url rewriting

[lmauldinpe15]

I have a single Nginx installation and I am using PHP-FPM to serve multiple
PHP applications in sub directories.  Example:

/var/www/  (this is 'root')
/var/www/a/foo/index.php
/var/www/a/bar/index.php
/var/www/b/bar/index.php

I want to setup url rewriting so that any request to
http://xxx/a/foo/index.php/users/login gets redirected to
http://xxx/a/foo/index.php and similarly
http://xxx/a/bar/index.php/users/login gets redirected to
http://xxx/a/bar/index.php

I may have a large number of applications in the sub directories so I don't
want to setup individual location blocks for each application.  Can I
accomplish this with a global rewrite rule?  Please let me know if you need
more information.

Luke

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?2,263024,263024#msg-263024

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: bug of discarding request body

[Maxim Dounin]

Hello!

On Mon, Nov 23, 2015 at 09:26:46AM -0800, Shuxin Yang wrote:

> Hi, Maxim:
> 
> Thank you very much for the comment, and sorry for my long previous
> email.
> 
> I guess you might misunderstand my previous email. Basically what I try
> to say
> is that the *OLD* bug (ticket/669 as you mentioned) is seen on the
> *PRISTINE* *NEW*
> 1.9.7 release. The attached script can reproduce the problem by simply
> invoke
> "./a.sh"
> 
>The a.sh donwload the 1.9.7 release, build it *without* any 3rd party
> module.
> 
>The problem is triggered by turning off unbuffered-uploading in proxy
> server.
> IIRC, when people report ticket/669, unbuffered-uploading was not available.

That's an old yet still unfixed bug.  Using unbuffered upload is 
just an easy way to trigger it.

>I guess we ngx_http_discard_request_body() should return NGX_AGAIN
> instead NGX_OK if discarding body is in progress.

No, it shouldn't.

[...]

> >Correct fix would be to stop nginx from re-using upstream
> >connections where a request wasn't completely sent.
> >
> We cannot tell if it is complete or not if ngx_http_discard_request_body()
> always
> returns NGX_OK

We can.  In frontend nginx, the one configured with upstream 
keepalive in your test.

Note well that the problem persists when not nginx but some other 
server is used as a backend.  And changing 
ngx_http_discard_request_body() behaviour will be useless.

-- 
Maxim Dounin
http://nginx.org/

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Поиск файла в двух папках

[Валентин Бартенев]

On Sunday 22 November 2015 17:38:20 Softer wrote:
> Добрый день. Я делал в такой же ситуации вот так:
> 
> location ~*
> \.(cur|jpg|jpeg|gif|png|zip|tgz|gz|rar|bz2|jar|lzma|exe|pdf|ppt|tar|swf|ico|flv|mp3|doc|xls|wav|bmp|rtf|txt|xml|docx|xlsx|woff2?)
> {
>  try_files $uri $uri/ @ph_static;
>  expires +1h;
> root /opt/www/old;
> }
> 
> location @ph_static
> {
> expires +1h;
> root /opt/www/new;
> }
> 

Более чем странное решение.

Если использовать именно try_files, то достаточно одного блока location
и директивы в нем:

  try_files /site_old$uri /site_new$uri =404;
  root /var/www/;

--
Валентин Бартенев
___
nginx-ru mailing list
nginx-ru@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-ru

Re: Complex url rewriting

[Aleksandar Lazic]

Hi.

Am 23-11-2015 18:48, schrieb lmauldinpe15:
I have a single Nginx installation and I am using PHP-FPM to serve 
multiple

PHP applications in sub directories.  Example:

/var/www/  (this is 'root')
/var/www/a/foo/index.php
/var/www/a/bar/index.php
/var/www/b/bar/index.php

I want to setup url rewriting so that any request to
http://xxx/a/foo/index.php/users/login gets redirected to
http://xxx/a/foo/index.php and similarly
http://xxx/a/bar/index.php/users/login gets redirected to
http://xxx/a/bar/index.php

I may have a large number of applications in the sub directories so I 
don't

want to setup individual location blocks for each application.  Can I
accomplish this with a global rewrite rule?  Please let me know if you 
need

more information.


How about to use the map module and some lines of scripting

http://nginx.org/en/docs/http/ngx_http_map_module.html

and use the right command for the location block.

http://nginx.org/en/docs/http/ngx_http_core_module.html#alias
http://nginx.org/en/docs/http/ngx_http_core_module.html#root

Pick a variable of your choice ;-)

http://nginx.org/en/docs/varindex.html
I would suggest

http://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_uri

but that's my opinion.

Do you a favor and use the debug log for debugging ;-)
http://nginx.org/en/docs/debugging_log.html

BR Aleks

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: bug of discarding request body

[Shuxin Yang]

Hi, Maxim:

   Thank you so much for your insightful comment!

   Unbuffered-uploading not just to make things easier to reproduce the 
problem.

It is trivial. So to speak.

   It is translating to say it is rather dangerous to use the 
unbuffered-uploading along
with keepalive connections, as the combination make the proxy server 
paper thin

to penetrate.

Thanks
Shuxin

On 11/23/2015 09:53 AM, Maxim Dounin wrote:

Hello!

On Mon, Nov 23, 2015 at 09:26:46AM -0800, Shuxin Yang wrote:


Hi, Maxim:

 Thank you very much for the comment, and sorry for my long previous
email.

 I guess you might misunderstand my previous email. Basically what I try
to say
is that the *OLD* bug (ticket/669 as you mentioned) is seen on the
*PRISTINE* *NEW*
1.9.7 release. The attached script can reproduce the problem by simply
invoke
"./a.sh"

The a.sh donwload the 1.9.7 release, build it *without* any 3rd party
module.

The problem is triggered by turning off unbuffered-uploading in proxy
server.
IIRC, when people report ticket/669, unbuffered-uploading was not available.

That's an old yet still unfixed bug.  Using unbuffered upload is
just an easy way to trigger it.


I guess we ngx_http_discard_request_body() should return NGX_AGAIN
instead NGX_OK if discarding body is in progress.

No, it shouldn't.

[...]


Correct fix would be to stop nginx from re-using upstream
connections where a request wasn't completely sent.


We cannot tell if it is complete or not if ngx_http_discard_request_body()
always
returns NGX_OK

We can.  In frontend nginx, the one configured with upstream
keepalive in your test.

Note well that the problem persists when not nginx but some other
server is used as a backend.  And changing
ngx_http_discard_request_body() behaviour will be useless.



___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

[nginx] Core: enabled "include" inside http upstreams (ticket #6...

[Ruslan Ermilov]

details:   http://hg.nginx.org/nginx/rev/18428f775b2c
branches:  
changeset: 6305:18428f775b2c
user:  Ruslan Ermilov 
date:  Mon Nov 23 12:40:19 2015 +0300
description:
Core: enabled "include" inside http upstreams (ticket #635).

The directive already works inside stream upstream blocks.

diffstat:

 src/core/ngx_conf_file.h |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff -r 520ec1917f1d -r 18428f775b2c src/core/ngx_conf_file.h
--- a/src/core/ngx_conf_file.h  Mon Nov 23 22:48:31 2015 +0300
+++ b/src/core/ngx_conf_file.h  Mon Nov 23 12:40:19 2015 +0300
@@ -50,7 +50,7 @@
 #define NGX_DIRECT_CONF  0x0001
 
 #define NGX_MAIN_CONF0x0100
-#define NGX_ANY_CONF 0x0F00
+#define NGX_ANY_CONF 0x1F00
 
 
 

___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel