Re: proxy_pass and concurrent connections

2018-02-28 Thread Дилян Палаузов 

Hello,

thanks for your answer.

The documentation is explicit when an upstream server is used with proxy_pass, 
but it says nothing, when proxy_pass is used with an URL.  Being explicit 
somewhere and stating nothing on the same manner in an alternative scenario 
leaves space for interpretations, hence my question.

The text "Warning: The ticket field 'nginx_version' is invalid: nginx_version is 
required " is misleading.  It is not a warning but a permanent error.

Moreover the 'Version' field was set, the message shall be "The 'nginx -V' field at 
the bottom is required" (or alternatively remove the Version field and leave only 
'nging -V', then it will be absolutely clear what is meant).

Regards
  Dilian

On 02/28/18 16:52, Maxim Dounin wrote:

Hello!

On Wed, Feb 28, 2018 at 11:52:18AM +0100, Дилян Палаузов wrote:


when I try to enter a bug at
https://trac.nginx.org/nginx/newticket#ticket, choose as version
1.12.x and submit the system rejects the ticket with the
message:

Warning: The ticket field 'nginx_version' is invalid:
nginx_version is required


That's about the "nginx -V" field you've left blank.


And here is the actual question:

proxy_pass can accept an URL or a server group. When a server
group (upstream) is defined, then with max_conns= the maximum
number of simultaneous connections can be specified.

The documentation of proxy_pass shall clarify, whether by
default with URL only consequent connections are allowed/whether
defining upstream is the only way to introduce parallelism
towards the proxy.


Or not, bacause the answer is obvious unless you have very strange
background.  Moreover, the documentation explicitly explains the
default for "max_conns", so this should be obvious regardless of
the background:

: max_conns=number
: limits the maximum number of simultaneous active connections to
: the proxied server (1.11.5). Default value is zero, meaning there
: is no limit.

If you for some reason think that only consequent connections are
allowed, most likely you are facing limitations of your backend.


___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

Re: [PATCH] $request_scheme variable

2018-02-28 Thread Maxim Dounin 
Hello!

On Tue, Feb 27, 2018 at 10:32:40PM +, Chris Branch via nginx-devel wrote:

> Hi, just giving this patch some birthday bumps.
> 
> > On 27 Feb 2017, at 11:58, Chris Branch via nginx-devel 
> >  wrote:
> > 
> > # HG changeset patch
> > # User Chris Branch 
> > # Date 1488195909 0
> > #  Mon Feb 27 11:45:09 2017 +
> > # Node ID 05f555d65a33ebf005fedc569fb52eba3758e1d7
> > # Parent  87cf6ddb41c216876d13cffa5e637a61b159362c
> > $request_scheme variable.
> > 
> > Contains the URI scheme supplied by the client. If no scheme supplied,
> > equivalent to $scheme.
> > 
> > Scheme can be supplied by the client in two ways:
> > 
> > * HTTP/2 :scheme pseudo-header.
> > * HTTP/1 absolute URI in request line.

The $scheme variable is already documented as

: $scheme
: request scheme, “http” or “https”

and introducing additional variable with the $request_scheme might 
not be a good idea.

If we really need this for some reason, we should rather consider 
changing $scheme instead, like we do with $host.  This might be a 
bad idea for other reasons though, as an ability to supply 
incorrect $scheme might cause security problems.

-- 
Maxim Dounin
http://mdounin.ru/
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

Re: Antw: [PATCH 1 of 2] Access log: Support for disabling escaping

2018-02-28 Thread Vladimir Homutov 
On Thu, Feb 22, 2018 at 06:12:52PM +0100, Johannes Baiter wrote:
> Sorry, I accidentally submitted an incomplete version of the patch.
> Here is the corrected version.
>

Hello,

I've slightly updated the patch (also note your mail client have broken
it - you may want to update settings to avoid this), please take a look.

See also ticket 1450: https://trac.nginx.org/nginx/ticket/1450

# HG changeset patch
# User Vladimir Homutov 
# Date 1519834295 -10800
#  Wed Feb 28 19:11:35 2018 +0300
# Node ID d420ce6b46768ea7eb23bdec84f992212293af19
# Parent  20f139e9ffa84f1a1db6039547cd35fc4534
Access log: support for disabling escaping (ticket #1450).

Based on patches by Johannes Baiter 
and Calin Don.

diff --git a/src/http/modules/ngx_http_log_module.c 
b/src/http/modules/ngx_http_log_module.c
--- a/src/http/modules/ngx_http_log_module.c
+++ b/src/http/modules/ngx_http_log_module.c
@@ -90,6 +90,11 @@ typedef struct {
 } ngx_http_log_var_t;
 
 
+#define NGX_HTTP_LOG_ESCAPE_DEFAULT  0
+#define NGX_HTTP_LOG_ESCAPE_JSON 1
+#define NGX_HTTP_LOG_ESCAPE_NONE 2
+
+
 static void ngx_http_log_write(ngx_http_request_t *r, ngx_http_log_t *log,
 u_char *buf, size_t len);
 static ssize_t ngx_http_log_script_write(ngx_http_request_t *r,
@@ -126,7 +131,7 @@ static u_char *ngx_http_log_request_leng
 ngx_http_log_op_t *op);
 
 static ngx_int_t ngx_http_log_variable_compile(ngx_conf_t *cf,
-ngx_http_log_op_t *op, ngx_str_t *value, ngx_uint_t json);
+ngx_http_log_op_t *op, ngx_str_t *value, ngx_uint_t escape);
 static size_t ngx_http_log_variable_getlen(ngx_http_request_t *r,
 uintptr_t data);
 static u_char *ngx_http_log_variable(ngx_http_request_t *r, u_char *buf,
@@ -136,6 +141,10 @@ static size_t ngx_http_log_json_variable
 uintptr_t data);
 static u_char *ngx_http_log_json_variable(ngx_http_request_t *r, u_char *buf,
 ngx_http_log_op_t *op);
+static size_t ngx_http_log_unescaped_variable_getlen(ngx_http_request_t *r,
+uintptr_t data);
+static u_char *ngx_http_log_unescaped_variable(ngx_http_request_t *r,
+u_char *buf, ngx_http_log_op_t *op);
 
 
 static void *ngx_http_log_create_main_conf(ngx_conf_t *cf);
@@ -905,7 +914,7 @@ ngx_http_log_request_length(ngx_http_req
 
 static ngx_int_t
 ngx_http_log_variable_compile(ngx_conf_t *cf, ngx_http_log_op_t *op,
-ngx_str_t *value, ngx_uint_t json)
+ngx_str_t *value, ngx_uint_t escape)
 {
 ngx_int_t  index;
 
@@ -916,11 +925,18 @@ ngx_http_log_variable_compile(ngx_conf_t
 
 op->len = 0;
 
-if (json) {
+switch (escape) {
+case NGX_HTTP_LOG_ESCAPE_JSON:
 op->getlen = ngx_http_log_json_variable_getlen;
 op->run = ngx_http_log_json_variable;
+break;
 
-} else {
+case NGX_HTTP_LOG_ESCAPE_NONE:
+op->getlen = ngx_http_log_unescaped_variable_getlen;
+op->run = ngx_http_log_unescaped_variable;
+break;
+
+default: /* NGX_HTTP_LOG_ESCAPE_DEFAULT */
 op->getlen = ngx_http_log_variable_getlen;
 op->run = ngx_http_log_variable;
 }
@@ -1073,6 +1089,39 @@ ngx_http_log_json_variable(ngx_http_requ
 }
 
 
+static size_t
+ngx_http_log_unescaped_variable_getlen(ngx_http_request_t *r, uintptr_t data)
+{
+ngx_http_variable_value_t  *value;
+
+value = ngx_http_get_indexed_variable(r, data);
+
+if (value == NULL || value->not_found) {
+return 0;
+}
+
+value->escape = 0;
+
+return value->len;
+}
+
+
+static u_char *
+ngx_http_log_unescaped_variable(ngx_http_request_t *r, u_char *buf,
+ngx_http_log_op_t *op)
+{
+ngx_http_variable_value_t  *value;
+
+value = ngx_http_get_indexed_variable(r, op->data);
+
+if (value == NULL || value->not_found) {
+return buf;
+}
+
+return ngx_cpymem(buf, value->data, value->len);
+}
+
+
 static void *
 ngx_http_log_create_main_conf(ngx_conf_t *cf)
 {
@@ -1536,18 +1585,21 @@ ngx_http_log_compile_format(ngx_conf_t *
 size_t   i, len;
 ngx_str_t   *value, var;
 ngx_int_t   *flush;
-ngx_uint_t   bracket, json;
+ngx_uint_t   bracket, escape;
 ngx_http_log_op_t   *op;
 ngx_http_log_var_t  *v;
 
-json = 0;
+escape = NGX_HTTP_LOG_ESCAPE_DEFAULT;
 value = args->elts;
 
 if (s < args->nelts && ngx_strncmp(value[s].data, "escape=", 7) == 0) {
 data = value[s].data + 7;
 
 if (ngx_strcmp(data, "json") == 0) {
-json = 1;
+escape = NGX_HTTP_LOG_ESCAPE_JSON;
+
+} else if (ngx_strcmp(data, "none") == 0) {
+escape = NGX_HTTP_LOG_ESCAPE_NONE;
 
 } else if (ngx_strcmp(data, "default") != 0) {
 ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
@@ -1636,7 +1688,7 @@ ngx_http_log_compile_format(ngx_conf_t *
 }
 }
 
-if (ngx_http_log_variable_compile(cf, op, , json)
+if (ngx_http_log_variable_compile(cf, op, ,

[njs] Skip empty buffers in HTTP response send().

2018-02-28 Thread Roman Arutyunyan 
details:   http://hg.nginx.org/njs/rev/c86a0cc40ce5
branches:  
changeset: 454:c86a0cc40ce5
user:  Roman Arutyunyan 
date:  Wed Feb 28 19:16:25 2018 +0300
description:
Skip empty buffers in HTTP response send().

Such buffers lead to send errors and should never be sent.

diffstat:

 nginx/ngx_http_js_module.c |  4 
 1 files changed, 4 insertions(+), 0 deletions(-)

diffs (14 lines):

diff -r ab1f67b69707 -r c86a0cc40ce5 nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.cWed Feb 28 16:20:11 2018 +0300
+++ b/nginx/ngx_http_js_module.cWed Feb 28 19:16:25 2018 +0300
@@ -891,6 +891,10 @@ ngx_http_js_ext_send(njs_vm_t *vm, njs_v
 return NJS_ERROR;
 }
 
+if (s.length == 0) {
+continue;
+}
+
 /* TODO: njs_value_release(vm, value) in buf completion */
 
 ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

Re: proxy_pass and concurrent connections

2018-02-28 Thread Maxim Dounin 
Hello!

On Wed, Feb 28, 2018 at 11:52:18AM +0100, Дилян Палаузов wrote:

> when I try to enter a bug at 
> https://trac.nginx.org/nginx/newticket#ticket, choose as version 
> 1.12.x and submit the system rejects the ticket with the 
> message:
> 
> Warning: The ticket field 'nginx_version' is invalid: 
> nginx_version is required

That's about the "nginx -V" field you've left blank.

> And here is the actual question:
> 
> proxy_pass can accept an URL or a server group. When a server 
> group (upstream) is defined, then with max_conns= the maximum 
> number of simultaneous connections can be specified.
> 
> The documentation of proxy_pass shall clarify, whether by 
> default with URL only consequent connections are allowed/whether 
> defining upstream is the only way to introduce parallelism 
> towards the proxy.

Or not, bacause the answer is obvious unless you have very strange 
background.  Moreover, the documentation explicitly explains the 
default for "max_conns", so this should be obvious regardless of 
the background:

: max_conns=number
: limits the maximum number of simultaneous active connections to 
: the proxied server (1.11.5). Default value is zero, meaning there 
: is no limit.

If you for some reason think that only consequent connections are 
allowed, most likely you are facing limitations of your backend.

-- 
Maxim Dounin
http://mdounin.ru/
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

[nginx] Generic subrequests in memory.

2018-02-28 Thread Roman Arutyunyan 
details:   http://hg.nginx.org/nginx/rev/20f139e9ffa8
branches:  
changeset: 7220:20f139e9ffa8
user:  Roman Arutyunyan 
date:  Wed Feb 28 16:56:58 2018 +0300
description:
Generic subrequests in memory.

Previously, only the upstream response body could be accessed with the
NGX_HTTP_SUBREQUEST_IN_MEMORY feature.  Now any response body from a subrequest
can be saved in a memory buffer.  It is available as a single buffer in r->out
and the buffer size is configured by the subrequest_output_buffer_size
directive.

Upstream, proxy and fastcgi code used to handle the old-style feature is
removed.

diffstat:

 src/http/modules/ngx_http_fastcgi_module.c|   30 --
 src/http/modules/ngx_http_proxy_module.c  |   30 --
 src/http/modules/ngx_http_ssi_filter_module.c |8 +-
 src/http/ngx_http_core_module.c   |   21 
 src/http/ngx_http_core_module.h   |2 +
 src/http/ngx_http_postpone_filter_module.c|   78 
 src/http/ngx_http_upstream.c  |  126 +-
 7 files changed, 107 insertions(+), 188 deletions(-)

diffs (428 lines):

diff -r d0d32b33167d -r 20f139e9ffa8 src/http/modules/ngx_http_fastcgi_module.c
--- a/src/http/modules/ngx_http_fastcgi_module.cThu Feb 22 17:25:43 
2018 +0300
+++ b/src/http/modules/ngx_http_fastcgi_module.cWed Feb 28 16:56:58 
2018 +0300
@@ -2512,36 +2512,6 @@ ngx_http_fastcgi_non_buffered_filter(voi
 break;
 }
 
-/* provide continuous buffer for subrequests in memory */
-
-if (r->subrequest_in_memory) {
-
-cl = u->out_bufs;
-
-if (cl) {
-buf->pos = cl->buf->pos;
-}
-
-buf->last = buf->pos;
-
-for (cl = u->out_bufs; cl; cl = cl->next) {
-ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
-   "http fastcgi in memory %p-%p %O",
-   cl->buf->pos, cl->buf->last, ngx_buf_size(cl->buf));
-
-if (buf->last == cl->buf->pos) {
-buf->last = cl->buf->last;
-continue;
-}
-
-buf->last = ngx_movemem(buf->last, cl->buf->pos,
-cl->buf->last - cl->buf->pos);
-
-cl->buf->pos = buf->last - (cl->buf->last - cl->buf->pos);
-cl->buf->last = buf->last;
-}
-}
-
 return NGX_OK;
 }
 
diff -r d0d32b33167d -r 20f139e9ffa8 src/http/modules/ngx_http_proxy_module.c
--- a/src/http/modules/ngx_http_proxy_module.c  Thu Feb 22 17:25:43 2018 +0300
+++ b/src/http/modules/ngx_http_proxy_module.c  Wed Feb 28 16:56:58 2018 +0300
@@ -2321,36 +2321,6 @@ ngx_http_proxy_non_buffered_chunked_filt
 return NGX_ERROR;
 }
 
-/* provide continuous buffer for subrequests in memory */
-
-if (r->subrequest_in_memory) {
-
-cl = u->out_bufs;
-
-if (cl) {
-buf->pos = cl->buf->pos;
-}
-
-buf->last = buf->pos;
-
-for (cl = u->out_bufs; cl; cl = cl->next) {
-ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
-   "http proxy in memory %p-%p %O",
-   cl->buf->pos, cl->buf->last, ngx_buf_size(cl->buf));
-
-if (buf->last == cl->buf->pos) {
-buf->last = cl->buf->last;
-continue;
-}
-
-buf->last = ngx_movemem(buf->last, cl->buf->pos,
-cl->buf->last - cl->buf->pos);
-
-cl->buf->pos = buf->last - (cl->buf->last - cl->buf->pos);
-cl->buf->last = buf->last;
-}
-}
-
 return NGX_OK;
 }
 
diff -r d0d32b33167d -r 20f139e9ffa8 
src/http/modules/ngx_http_ssi_filter_module.c
--- a/src/http/modules/ngx_http_ssi_filter_module.c Thu Feb 22 17:25:43 
2018 +0300
+++ b/src/http/modules/ngx_http_ssi_filter_module.c Wed Feb 28 16:56:58 
2018 +0300
@@ -2231,9 +2231,11 @@ ngx_http_ssi_set_variable(ngx_http_reque
 {
 ngx_str_t  *value = data;
 
-if (r->upstream) {
-value->len = r->upstream->buffer.last - r->upstream->buffer.pos;
-value->data = r->upstream->buffer.pos;
+if (r->headers_out.status < NGX_HTTP_SPECIAL_RESPONSE
+&& r->out && r->out->buf)
+{
+value->len = r->out->buf->last - r->out->buf->pos;
+value->data = r->out->buf->pos;
 }
 
 return rc;
diff -r d0d32b33167d -r 20f139e9ffa8 src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c   Thu Feb 22 17:25:43 2018 +0300
+++ b/src/http/ngx_http_core_module.c   Wed Feb 28 16:56:58 2018 +0300
@@ -399,6 +399,13 @@ static ngx_command_t  ngx_http_core_comm
   offsetof(ngx_http_core_loc_conf_t, sendfile_max_chunk),
   NULL },
 
+{ ngx_string("subrequest_output_buffer_size"),
+  NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
+  ngx_conf_set_size_slot,
+  NGX_HTTP_LOC_CONF_OFFSET,
+

proxy_pass and concurrent connections

2018-02-28 Thread Дилян Палаузов 

Hello,

when I try to enter a bug at https://trac.nginx.org/nginx/newticket#ticket, 
choose as version 1.12.x and submit the system rejects the ticket with the 
message:

Warning: The ticket field 'nginx_version' is invalid: nginx_version is required

And here is the actual question:

proxy_pass can accept an URL or a server group. When a server group (upstream) 
is defined, then with max_conns= the maximum number of simultaneous connections 
can be specified.

The documentation of proxy_pass shall clarify, whether by default with URL only 
consequent connections are allowed/whether defining upstream is the only way to 
introduce parallelism towards the proxy.

Regards
  Dilian
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

[njs] Fixed String.prototype.toUTF8() function.

2018-02-28 Thread Igor Sysoev 
details:   http://hg.nginx.org/njs/rev/ab1f67b69707
branches:  
changeset: 453:ab1f67b69707
user:  Igor Sysoev 
date:  Wed Feb 28 16:20:11 2018 +0300
description:
Fixed String.prototype.toUTF8() function.

A byte string returned by String.prototype.toUTF8() had length equal
to its size so the string can be processed later as an ASCII string.

diffstat:

 njs/njs_string.c |  5 +
 njs/test/njs_unit_test.c |  9 +
 2 files changed, 14 insertions(+), 0 deletions(-)

diffs (34 lines):

diff -r 0f1c3efcd894 -r ab1f67b69707 njs/njs_string.c
--- a/njs/njs_string.c  Tue Feb 27 14:11:00 2018 +0300
+++ b/njs/njs_string.c  Wed Feb 28 16:20:11 2018 +0300
@@ -1051,6 +1051,11 @@ njs_string_slice(njs_vm_t *vm, njs_value
 start += slice->start;
 size = slice->length;
 
+if (string->length == 0) {
+/* Byte string. */
+length = 0;
+}
+
 } else {
 /* UTF-8 string. */
 end = start + string->size;
diff -r 0f1c3efcd894 -r ab1f67b69707 njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c  Tue Feb 27 14:11:00 2018 +0300
+++ b/njs/test/njs_unit_test.c  Wed Feb 28 16:20:11 2018 +0300
@@ -3529,6 +3529,15 @@ static njs_unit_test_t  njs_test[] =
 { nxt_string("'α'.toUTF8()[0]"),
   nxt_string("\xCE") },
 
+{ nxt_string("/^\\x80$/.test('\\x80'.toBytes())"),
+  nxt_string("true") },
+
+{ nxt_string("/^\\xC2\\x80$/.test('\\x80'.toUTF8())"),
+  nxt_string("true") },
+
+{ nxt_string("'α'.toUTF8().toBytes()"),
+  nxt_string("α") },
+
 { nxt_string("var a = 'a'.toBytes() + 'α'; a + a.length"),
   nxt_string("aα3") },
 
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel