OpenTracing Module

[rnburn]

Hello Everyone,

I've been working on an NGINX module
[github.com/rnburn/nginx-opentracing] to add support for OpenTracing
[http://opentracing.io]. It uses OpenTracing's C++ API
[http://github.com/opentracing/opentracing-cpp] and attaches handlers
to the NGX_HTTP_PREACCESS_PHASE  and NGX_HTTP_LOG_PHASE phases to
start/stop spans to track requests handled.

It currently supports LightStep's C++ tracer
[github.com/lightstep/lightstep-tracer-cpp] and a C++ version of
Zipkin's tracer [github.com/rnburn/zipkin-cpp-opentracing].

I put together a simple example
[github.com/rnburn/nginx-opentracing/tree/master/example/trivial] that
shows it interoperating with a Go server traced with Zipkin and wrote
up a description for a more complicated example
[github.com/rnburn/nginx-opentracing/blob/master/doc/Tutorial.md]
showing it working with a Node server using LightStep.

If anyone would like to try it or has any feedback, let me know.
Ryan

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?2,276370,276370#msg-276370

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Secure Link Md5 Implementation

[Francis Daly]

On Wed, Sep 13, 2017 at 08:05:14AM -0400, anish10dec wrote:

Hi there,

> How to use two secret Keys for Secure Link Md5.

the stock nginx secure_link module does not support multiple/alternate keys.

I have not tested, but I guess that maybe in your primary 'if
($secure_link = "") {' section you could "rewrite ^ /ex/$uri;" and then
have a new location{} that matches those requests and uses the secondary
secret key in its config.

(Or you could rewrite the module to do what you want; but that is probably
much more work.)

Good luck with it,

f
-- 
Francis Dalyfran...@daoine.org
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Support Request for Preserving Proxied server original Chunk sizes

[Jaradeh, Fares]

Hello

I am contacting you for support regarding
https://stackoverflow.com/questions/46165415/configure-nginx-proxy-to-preserve-the-chunk-size-sent-from-the-proxied-backend-s

While trying to use Nginx-Ingress-controller in K8s, ran into issue related to 
the behavior of nginx proxy,

The proxied backend server (our NGW) is set to return audio in chunks of http 
to the client with multiparts & Some of our clients (already in the field) have 
so far expected every single HTTP chunk to contain a full part of an audio 
segmentie a single audio segment may not be broken across several HTTP 
chunks.

For our use case, we had to set the  proxy_buffering : off so to allow the 
nginx to send back results as soon as possible to not increase the CPL, But as 
a side effect, we noticed that the nginx was NOT trying to preserve the chunks 
sizes sent back by the proxied server (NGW), which would cause clients in the 
field to break.

So a single Audio part that is sent back by the NGW with a single http chunk of 
size 24838 may be broken into 5 or 6 chunks returned by nginx-proxy to the 
client  with the total size amounting to the same thing We suspect this is 
due to the speed to read/write of the responses on Nginx, and it may read a 
single chunk in separate calls (example 12000 bytes then another 11000 bytes 
then another 1838 bytes)  and because buffering is OFF...these things are being 
sent back as independent chunks to the client.

We understand that the behavior of nginx is fully compliant with http chunking 
and that the client app should be better implemented to not assume coupling of 
chunks and audio segments full parts, BUT, customers app already in the field 
cannot be helped,

Enabling proxy_buffering : on is not an option, as it delay responses until ALL 
the audio segments are ready (from all chunks) and it breaks the 1chunk to 1 
audio segment rule as well.

Could you please advise if there is any way to control the nginx buffering to 
allow it to buffer per Chunk, ir do not send back received data unless a full 
Http chunk is read ?

Thank you for you support

Regards

Fares


___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: [EXTERNAL] Re: Preserving Proxied server original Chunk sizes

[Maxim Dounin]

Hello!

On Wed, Sep 13, 2017 at 07:56:59PM +, Jaradeh, Fares wrote:

> Thank you for replying to confirm my message was received, 
> 
> Would you be able to advise us regarding the issue in attached ...
> 
> Please note, our chunk sizes are dynamic depending on the request

As previously suggested, please use the nginx@ mailing list for
user-level questions.  See http://nginx.org/en/support.html for 
more information.

[...]

-- 
Maxim Dounin
http://nginx.org/
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

Re: "worker process is shutting down" и ошибки open files

[Maxim Dounin]

Hello!

On Wed, Sep 13, 2017 at 03:38:55PM -0400, ingtar wrote:

> Maxim Dounin Wrote:
> ---
> 
> > Такое может произойти, например, если nginx упёрся в ограничение 
> > по количеству открытых файлов на пользователя, и в этот момент 
> > попытались сделать reload.
> 
> Имеется в виду глобальные ограничения в /etc/security/limits.conf или в
> worker_rlimit_nofile?

Директива worker_rlimit_nofile позволяет увеличить ограничение на 
количество открытых файлов для рабочих процессов.  Для мастера при 
этом действуют ограничения, установленные системой (обычно это 
_не_ limits.conf - он примеряется только для пользовательских 
сессий, а не для демонов).  Наступить можно и на то, и на другое.

-- 
Maxim Dounin
http://nginx.org/
___
nginx-ru mailing list
nginx-ru@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-ru

Re: "worker process is shutting down" и ошибки open files

[ingtar]

И еще вопрос - может ли мастер процесс сам перезапускать воркеров через
какое-то время?

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?21,276356,276364#msg-276364

___
nginx-ru mailing list
nginx-ru@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-ru

RE: [EXTERNAL] Re: Preserving Proxied server original Chunk sizes

[Jaradeh, Fares]

Hello, 

Thank you for replying to confirm my message was received, 

Would you be able to advise us regarding the issue in attached ...

Please note, our chunk sizes are dynamic depending on the request

Thank you 

Fares

-Original Message-
From: nginx-devel [mailto:nginx-devel-boun...@nginx.org] On Behalf Of Maxim 
Dounin
Sent: Wednesday, September 13, 2017 3:51 PM
To: nginx-devel@nginx.org
Subject: [EXTERNAL] Re: Preserving Proxied server original Chunk sizes

Hello!

On Wed, Sep 13, 2017 at 07:09:19PM +, Jaradeh, Fares wrote:

> I am contacting you for support regarding 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__stackoverflow.com
> _questions_46165415_configure-2Dnginx-2Dproxy-2Dto-2Dpreserve-2Dthe-2D
> chunk-2Dsize-2Dsent-2Dfrom-2Dthe-2Dproxied-2Dbackend-2Ds=DwICAg=dj
> jh8EKwHtOepW4Bjau0lKhLlu-DxM1dlgP0rrLsOzY=hzyZG1oadm3bN7GeqSIg8qI6a6
> lJEPnNOjJY5iD0d1w=ZvrthZdjbKlytjVnP_izUy7sTxjmzyMLCeyo_TkfiXc=8Usn
> jvVLJTkiE2OBN4t1415kzWOpBFMW3Lpc4hn2LOo=

[...]

This mailing list is dedicated to nginx development.  For questions about using 
nginx please use the nginx@ mailing list instead.  See 
https://urldefense.proofpoint.com/v2/url?u=http-3A__nginx.org_en_support.html=DwICAg=djjh8EKwHtOepW4Bjau0lKhLlu-DxM1dlgP0rrLsOzY=hzyZG1oadm3bN7GeqSIg8qI6a6lJEPnNOjJY5iD0d1w=ZvrthZdjbKlytjVnP_izUy7sTxjmzyMLCeyo_TkfiXc=cArXUByJZlae56EUV_ri_WiVUOCo1AqxOXW-HjYIknk=
  for more information.  Thank you.

--
Maxim Dounin
https://urldefense.proofpoint.com/v2/url?u=http-3A__nginx.org_=DwICAg=djjh8EKwHtOepW4Bjau0lKhLlu-DxM1dlgP0rrLsOzY=hzyZG1oadm3bN7GeqSIg8qI6a6lJEPnNOjJY5iD0d1w=ZvrthZdjbKlytjVnP_izUy7sTxjmzyMLCeyo_TkfiXc=gqprp8uh21K4xaEsZuNL-K8Yunocdey2m5Km2hy-xws=
___
nginx-devel mailing list
nginx-devel@nginx.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.nginx.org_mailman_listinfo_nginx-2Ddevel=DwICAg=djjh8EKwHtOepW4Bjau0lKhLlu-DxM1dlgP0rrLsOzY=hzyZG1oadm3bN7GeqSIg8qI6a6lJEPnNOjJY5iD0d1w=ZvrthZdjbKlytjVnP_izUy7sTxjmzyMLCeyo_TkfiXc=r0WC5Dre8WukhPB1owslPpATjC8vWw_viFiARvSBUsA=
 
--- Begin Message ---




Hello



I am contacting you for support regarding

https://stackoverflow.com/questions/46165415/configure-nginx-proxy-to-preserve-the-chunk-size-sent-from-the-proxied-backend-s



While trying to use Nginx-Ingress-controller in K8s, ran into issue related to 
the behavior of nginx proxy,



The proxied backend server (our NGW) is set to return audio in chunks of http 
to the client with multiparts & Some of our clients (already in the field) have 
so far expected every single HTTP chunk to contain a full part of an audio 
segmentie a single audio segment may not be broken across several HTTP 
chunks.



For our use case, we had to set the  proxy_buffering : off so to allow the 
nginx to send back results as soon as possible to not increase the CPL, But as 
a side effect, we noticed that the nginx was NOT trying to preserve the chunks 
sizes sent back by the proxied server (NGW), which would cause clients in the 
field to break.



So a single Audio part that is sent back by the NGW with a single http chunk of 
size 24838 may be broken into 5 or 6 chunks returned by nginx-proxy to the 
client  with the total size amounting to the same thing We suspect this is 
due to the speed to read/write of the responses on Nginx, and it may read a 
single chunk in separate calls (example 12000 bytes then another 11000 bytes 
then another 1838 bytes)  and because buffering is OFF...these things are being 
sent back as independent chunks to the client.



We understand that the behavior of nginx is fully compliant with http chunking 
and that the client app should be better implemented to not assume coupling of 
chunks and audio segments full parts, BUT, customers app already in the field 
cannot be helped,



Enabling proxy_buffering : on is not an option, as it delay responses until ALL 
the audio segments are ready (from all chunks) and it breaks the 1chunk to 1 
audio segment rule as well.



Could you please advise if there is any way to control the nginx buffering to 
allow it to buffer per Chunk, ir do not send back received data unless a full 
Http chunk is read ?



Thank you for you support



Regards



Fares





--- End Message ---
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

Re: Preserving Proxied server original Chunk sizes

[Maxim Dounin]

Hello!

On Wed, Sep 13, 2017 at 07:09:19PM +, Jaradeh, Fares wrote:

> I am contacting you for support regarding
> https://stackoverflow.com/questions/46165415/configure-nginx-proxy-to-preserve-the-chunk-size-sent-from-the-proxied-backend-s

[...]

This mailing list is dedicated to nginx development.  For 
questions about using nginx please use the nginx@ mailing list 
instead.  See http://nginx.org/en/support.html for more 
information.  Thank you.

-- 
Maxim Dounin
http://nginx.org/
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

Re: "worker process is shutting down" и ошибки open files

[ingtar]

Maxim Dounin Wrote:
---

> Такое может произойти, например, если nginx упёрся в ограничение 
> по количеству открытых файлов на пользователя, и в этот момент 
> попытались сделать reload.

Имеется в виду глобальные ограничения в /etc/security/limits.conf или в
worker_rlimit_nofile?

> Отмечу в скобках, что для восстановления из подобной ситуации - 
> обязательно перезапустить nginx.  Попытки делать очередные 
> reload'ы ни к чему хорошему не приведут.
> 

Да, перезапускаем.

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?21,276356,276361#msg-276361

___
nginx-ru mailing list
nginx-ru@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-ru

Re: "worker process is shutting down" и ошибки open files

[Maxim Dounin]

Hello!

On Wed, Sep 13, 2017 at 12:07:49PM -0400, ingtar wrote:

> Добрый день! Такая непонятная штука стала происходить (время появления не
> зафиксировали)
> Есть nginx в котором:
> worker_processes  28;
> worker_rlimit_nofile 2;
> 
> В случае выполнения nignx reload старые ооочень долго не завершаются.
> Настройка worker_shutdown_timeout 1m выставлена, но особо не помогает.
> Через какое-то время появляются ошибки Too many open files. Если посмотреть
> по открытым файлам для каждого процесса то картина следующая:

[...]

Для начала я бы рекомендовал посмотреть в логи ошибок, нет ли там 
сообщений с уровнями "crit", "alert" или "emerg", особенно - от 
мастера.  Если они есть - то разбираться, что их вызвало.

Вообще обращают на себя внимание два фактора:

- работающих рабочих процессов существенно меньше, чем должно быть 
  исходя из конфига;

- время запуска процессов распределено хаотически (в частности, 
  совпадает время запуска работающих процессов и завершающихся).

Это позволяет предположить, что нарушена коммуникация между 
мастером и рабочими процессами.  И, вероятно, были фатальные 
ошибки при запуске рабочих процессов, из-за которых они больше не 
могут запускаться.

Такое может произойти, например, если nginx упёрся в ограничение 
по количеству открытых файлов на пользователя, и в этот момент 
попытались сделать reload.

Отмечу в скобках, что для восстановления из подобной ситуации - 
обязательно перезапустить nginx.  Попытки делать очередные 
reload'ы ни к чему хорошему не приведут.

-- 
Maxim Dounin
http://nginx.org/
___
nginx-ru mailing list
nginx-ru@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-ru

Preserving Proxied server original Chunk sizes

[Jaradeh, Fares]

Hello

I am contacting you for support regarding
https://stackoverflow.com/questions/46165415/configure-nginx-proxy-to-preserve-the-chunk-size-sent-from-the-proxied-backend-s

While trying to use Nginx-Ingress-controller in K8s, ran into issue related to 
the behavior of nginx proxy,

The proxied backend server (our NGW) is set to return audio in chunks of http 
to the client with multiparts & Some of our clients (already in the field) have 
so far expected every single HTTP chunk to contain a full part of an audio 
segmentie a single audio segment may not be broken across several HTTP 
chunks.

For our use case, we had to set the  proxy_buffering : off so to allow the 
nginx to send back results as soon as possible to not increase the CPL, But as 
a side effect, we noticed that the nginx was NOT trying to preserve the chunks 
sizes sent back by the proxied server (NGW), which would cause clients in the 
field to break.

So a single Audio part that is sent back by the NGW with a single http chunk of 
size 24838 may be broken into 5 or 6 chunks returned by nginx-proxy to the 
client  with the total size amounting to the same thing We suspect this is 
due to the speed to read/write of the responses on Nginx, and it may read a 
single chunk in separate calls (example 12000 bytes then another 11000 bytes 
then another 1838 bytes)  and because buffering is OFF...these things are being 
sent back as independent chunks to the client.

We understand that the behavior of nginx is fully compliant with http chunking 
and that the client app should be better implemented to not assume coupling of 
chunks and audio segments full parts, BUT, customers app already in the field 
cannot be helped,

Enabling proxy_buffering : on is not an option, as it delay responses until ALL 
the audio segments are ready (from all chunks) and it breaks the 1chunk to 1 
audio segment rule as well.

Could you please advise if there is any way to control the nginx buffering to 
allow it to buffer per Chunk, ir do not send back received data unless a full 
Http chunk is read ?

Thank you for you support

Regards

Fares


___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

Re: "worker process is shutting down" и ошибки open files

[ingtar]

Да, 

proxy_cache_path   /lib/init/rw/proxy_cache  levels=1:2  
keys_zone=static:10m max_size=100m;

Завтра попробую посмотреть дебаг логи

Vasiliy P. Melnik Wrote:
---
> кеш используется ?
>

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?21,276356,276358#msg-276358

___
nginx-ru mailing list
nginx-ru@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-ru

Re: "worker process is shutting down" и ошибки open files

[Vasiliy P. Melnik]

кеш используется ?

13 сентября 2017 г., 19:07 пользователь ingtar 
написал:

> Добрый день! Такая непонятная штука стала происходить (время появления не
> зафиксировали)
> Есть nginx в котором:
> worker_processes  28;
> worker_rlimit_nofile 2;
>
> В случае выполнения nignx reload старые ооочень долго не завершаются.
> Настройка worker_shutdown_timeout 1m выставлена, но особо не помогает.
> Через какое-то время появляются ошибки Too many open files. Если посмотреть
> по открытым файлам для каждого процесса то картина следующая:
>
> несколько
> nginx: worker process
> Limit Soft Limit   Hard Limit   Units
>
> Max open files22files
>
> pid 43587
> Currently open files: 12510
>
> и очень много:
>
> nginx: worker process is shutting down
> Limit Soft Limit   Hard Limit   Units
>
> Max open files1524015240files
>
> pid 35319
> Currently open files: 217
>
> nginx: worker process is shutting down
> Limit Soft Limit   Hard Limit   Units
>
> Max open files1524015240files
>
> pid 35311
> Currently open files: 223
>
> Если посмотреть список процессов, то видно что нормальных воркеров мало, у
> каждого перекос по файлам и такие ошибки.
>
> nobody1752 18.0  5.2 2752480 1712492 ? S18:40   4:13 nginx:
> worker process is shutting down
> nobody6034 36.3  5.2 2752480 1740768 ? S17:38  30:45 nginx:
> worker process is shutting down
> nobody6035  8.1  4.2 2752480 1399368 ? S17:38   6:54 nginx:
> worker process is shutting down
> nobody6040  5.4  4.1 2752480 1359280 ? S17:38   4:35 nginx:
> worker process is shutting down
> nobody6049 12.5  4.6 2752480 1539268 ? S17:38  10:35 nginx:
> worker process is shutting down
> nobody6050 31.3  5.2 2752480 1739804 ? S17:38  26:31 nginx:
> worker process is shutting down
> nobody6052  3.3  4.0 2752480 1343572 ? S17:38   2:51 nginx:
> worker process is shutting down
> nobody6054  5.3  4.1 2752480 1360352 ? S17:38   4:33 nginx:
> worker process is shutting down
> nobody6055 26.2  5.4 2752480 1790544 ? S17:38  22:13 nginx:
> worker process is shutting down
> nobody6058  0.0  3.7 2752480 1227076 ? S17:38   0:03 nginx:
> cache manager process
> root  6168  0.0  3.9 2752480 1293176 ? Ss   13:24   0:16 nginx:
> master process /usr/sbin/nginx -g daemon on; master_process on;
> nobody   10656 48.4  5.2 2752480 1719536 ? S18:45   8:36 nginx:
> worker process is shutting down
> nobody   10657 31.1  4.4 2752480 1460292 ? S18:45   5:31 nginx:
> worker process is shutting down
> nobody   10658 70.8  5.2 2752480 1739444 ? R18:45  12:33 nginx:
> worker process
> nobody   10659 70.7  5.2 2752480 1739476 ? R18:45  12:32 nginx:
> worker process
> nobody   10660 10.7  4.1 2752480 1377328 ? S18:45   1:54 nginx:
> worker process is shutting down
> nobody   11053 25.3  4.3 2752480 1440108 ? S18:45   4:26 nginx:
> worker process is shutting down
> nobody   11054 51.3  5.3 2752480 1756036 ? S18:45   8:59 nginx:
> worker process is shutting down
> nobody   11055  2.2  4.0 2752480 1324696 ? S18:45   0:23 nginx:
> worker process is shutting down
> nobody   11056 38.6  4.8 2752480 1604476 ? S18:45   6:45 nginx:
> worker process is shutting down
> nobody   11057 17.9  4.2 2752480 1400344 ? S18:45   3:07 nginx:
> worker process is shutting down
> nobody   16049  9.6  4.1 2752480 1373540 ? S18:48   1:26 nginx:
> worker process is shutting down
> nobody   19782 29.2  5.4 2752480 1784320 ? S18:18  13:08 nginx:
> worker process is shutting down
> nobody   29881  7.8  4.7 2752480 1571664 ? S18:24   3:06 nginx:
> worker process is shutting down
> nobody   32145 22.2  5.3 2752480 1757248 ? S17:53  15:40 nginx:
> worker process is shutting down
> nobody   33168 19.8  5.3 2752480 1748384 ? S18:25   7:33 nginx:
> worker process is shutting down
> nobody   36049 94.5  5.1 2752480 1702212 ? R18:59   3:29 nginx:
> worker process
> nobody   40384 14.9  4.1 2489628 1365948 ? S16:22  24:07 nginx:
> worker process is shutting down
> root 43195  0.0  0.0  13956  2188 pts/0S+   19:03   0:00 grep
> --color=auto nginx
> nobody   45130 22.6  5.2 2752480 1715320 ? S18:32   6:58 nginx:
> worker process is shutting down
> nobody   48277  3.8  4.1 2489628 1369324 ? S16:58   4:48 nginx:
> worker process is shutting down
> nobody   49924  6.1  4.2 2489628 1411852 ? S16:59   7:37 nginx:
> worker process is shutting down
> nobody   49931  2.7  4.1 2489628 1368808 ? S16:59   3:22 nginx:
> worker process is shutting down
> nobody   49943  8.3  4.4 2489628 1453684 ? S

"worker process is shutting down" и ошибки open files

[ingtar]

Добрый день! Такая непонятная штука стала происходить (время появления не
зафиксировали)
Есть nginx в котором:
worker_processes  28;
worker_rlimit_nofile 2;

В случае выполнения nignx reload старые ооочень долго не завершаются.
Настройка worker_shutdown_timeout 1m выставлена, но особо не помогает.
Через какое-то время появляются ошибки Too many open files. Если посмотреть
по открытым файлам для каждого процесса то картина следующая:

несколько 
nginx: worker process
Limit Soft Limit   Hard Limit   Units   
 
Max open files22files   
 
pid 43587
Currently open files: 12510

и очень много:

nginx: worker process is shutting down
Limit Soft Limit   Hard Limit   Units   
 
Max open files1524015240files   
 
pid 35319
Currently open files: 217

nginx: worker process is shutting down
Limit Soft Limit   Hard Limit   Units   
 
Max open files1524015240files   
 
pid 35311
Currently open files: 223

Если посмотреть список процессов, то видно что нормальных воркеров мало, у
каждого перекос по файлам и такие ошибки.

nobody1752 18.0  5.2 2752480 1712492 ? S18:40   4:13 nginx:
worker process is shutting down  
nobody6034 36.3  5.2 2752480 1740768 ? S17:38  30:45 nginx:
worker process is shutting down  
nobody6035  8.1  4.2 2752480 1399368 ? S17:38   6:54 nginx:
worker process is shutting down  
nobody6040  5.4  4.1 2752480 1359280 ? S17:38   4:35 nginx:
worker process is shutting down  
nobody6049 12.5  4.6 2752480 1539268 ? S17:38  10:35 nginx:
worker process is shutting down  
nobody6050 31.3  5.2 2752480 1739804 ? S17:38  26:31 nginx:
worker process is shutting down  
nobody6052  3.3  4.0 2752480 1343572 ? S17:38   2:51 nginx:
worker process is shutting down  
nobody6054  5.3  4.1 2752480 1360352 ? S17:38   4:33 nginx:
worker process is shutting down  
nobody6055 26.2  5.4 2752480 1790544 ? S17:38  22:13 nginx:
worker process is shutting down  
nobody6058  0.0  3.7 2752480 1227076 ? S17:38   0:03 nginx:
cache manager process
root  6168  0.0  3.9 2752480 1293176 ? Ss   13:24   0:16 nginx:
master process /usr/sbin/nginx -g daemon on; master_process on;
nobody   10656 48.4  5.2 2752480 1719536 ? S18:45   8:36 nginx:
worker process is shutting down  
nobody   10657 31.1  4.4 2752480 1460292 ? S18:45   5:31 nginx:
worker process is shutting down  
nobody   10658 70.8  5.2 2752480 1739444 ? R18:45  12:33 nginx:
worker process   
nobody   10659 70.7  5.2 2752480 1739476 ? R18:45  12:32 nginx:
worker process   
nobody   10660 10.7  4.1 2752480 1377328 ? S18:45   1:54 nginx:
worker process is shutting down  
nobody   11053 25.3  4.3 2752480 1440108 ? S18:45   4:26 nginx:
worker process is shutting down  
nobody   11054 51.3  5.3 2752480 1756036 ? S18:45   8:59 nginx:
worker process is shutting down  
nobody   11055  2.2  4.0 2752480 1324696 ? S18:45   0:23 nginx:
worker process is shutting down  
nobody   11056 38.6  4.8 2752480 1604476 ? S18:45   6:45 nginx:
worker process is shutting down  
nobody   11057 17.9  4.2 2752480 1400344 ? S18:45   3:07 nginx:
worker process is shutting down  
nobody   16049  9.6  4.1 2752480 1373540 ? S18:48   1:26 nginx:
worker process is shutting down  
nobody   19782 29.2  5.4 2752480 1784320 ? S18:18  13:08 nginx:
worker process is shutting down  
nobody   29881  7.8  4.7 2752480 1571664 ? S18:24   3:06 nginx:
worker process is shutting down  
nobody   32145 22.2  5.3 2752480 1757248 ? S17:53  15:40 nginx:
worker process is shutting down  
nobody   33168 19.8  5.3 2752480 1748384 ? S18:25   7:33 nginx:
worker process is shutting down  
nobody   36049 94.5  5.1 2752480 1702212 ? R18:59   3:29 nginx:
worker process   
nobody   40384 14.9  4.1 2489628 1365948 ? S16:22  24:07 nginx:
worker process is shutting down  
root 43195  0.0  0.0  13956  2188 pts/0S+   19:03   0:00 grep
--color=auto nginx
nobody   45130 22.6  5.2 2752480 1715320 ? S18:32   6:58 nginx:
worker process is shutting down  
nobody   48277  3.8  4.1 2489628 1369324 ? S16:58   4:48 nginx:
worker process is shutting down  
nobody   49924  6.1  4.2 2489628 1411852 ? S16:59   7:37 nginx:
worker process is shutting down  
nobody   49931  2.7  4.1 2489628 1368808 ? S16:59   3:22 nginx:
worker 

[nginx] Introduced time truncation to December 31, 9999 (ticket #1368).

[Maxim Dounin]

details:   http://hg.nginx.org/nginx/rev/cdbcb73239ee
branches:  
changeset: 7104:cdbcb73239ee
user:  Maxim Dounin 
date:  Wed Sep 13 15:53:19 2017 +0300
description:
Introduced time truncation to December 31,  (ticket #1368).

Various buffers are allocated in an assumption that there would be
no more than 4 year digits.  This might not be true on platforms
with 64-bit time_t, as 64-bit time_t is able to represent more than that.
Such dates with more than 4 year digits hardly make sense though, as
various date formats in use do not allow them anyway.

As such, all dates are now truncated by ngx_gmtime() to December 31, .
This should have no effect on valid dates, though will prevent potential
buffer overflows on invalid ones.

diffstat:

 src/core/ngx_times.c |  10 ++
 1 files changed, 10 insertions(+), 0 deletions(-)

diffs (20 lines):

diff --git a/src/core/ngx_times.c b/src/core/ngx_times.c
--- a/src/core/ngx_times.c
+++ b/src/core/ngx_times.c
@@ -311,6 +311,16 @@ ngx_gmtime(time_t t, ngx_tm_t *tp)
 days = t / 86400;
 sec = t % 86400;
 
+/*
+ * no more than 4 year digits supported,
+ * truncate to December 31, , 23:59:59
+ */
+
+if (days > 2932896) {
+days = 2932896;
+sec = 86399;
+}
+
 /* January 1, 1970 was Thursday */
 
 wday = (4 + days) % 7;
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

[nginx] Style.

[Maxim Dounin]

details:   http://hg.nginx.org/nginx/rev/b38a8f0ca4a2
branches:  
changeset: 7101:b38a8f0ca4a2
user:  Maxim Dounin 
date:  Wed Sep 13 15:51:52 2017 +0300
description:
Style.

diffstat:

 src/core/ngx_parse_time.c |  5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diffs (21 lines):

diff --git a/src/core/ngx_parse_time.c b/src/core/ngx_parse_time.c
--- a/src/core/ngx_parse_time.c
+++ b/src/core/ngx_parse_time.c
@@ -44,14 +44,15 @@ ngx_parse_http_time(u_char *value, size_
 }
 }
 
-for (p++; p < end; p++)
+for (p++; p < end; p++) {
 if (*p != ' ') {
 break;
 }
+}
 
 if (end - p < 18) {
 return NGX_ERROR;
-}
+}
 
 if (fmt != isoc) {
 if (*p < '0' || *p > '9' || *(p + 1) < '0' || *(p + 1) > '9') {
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

[nginx] Fixed reference to time parsing code after 8b6fa4842133.

[Maxim Dounin]

details:   http://hg.nginx.org/nginx/rev/63699a40e2ff
branches:  
changeset: 7102:63699a40e2ff
user:  Maxim Dounin 
date:  Wed Sep 13 15:51:58 2017 +0300
description:
Fixed reference to time parsing code after 8b6fa4842133.

diffstat:

 src/core/ngx_times.c |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff --git a/src/core/ngx_times.c b/src/core/ngx_times.c
--- a/src/core/ngx_times.c
+++ b/src/core/ngx_times.c
@@ -320,7 +320,7 @@ ngx_gmtime(time_t t, ngx_tm_t *tp)
 
 /*
  * the algorithm based on Gauss' formula,
- * see src/http/ngx_http_parse_time.c
+ * see src/core/ngx_parse_time.c
  */
 
 /* days since March 1, 1 BC */
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

[nginx] Fixed ngx_gmtime() on 32-bit platforms with 64-bit time_t.

[Maxim Dounin]

details:   http://hg.nginx.org/nginx/rev/644d0457782a
branches:  
changeset: 7103:644d0457782a
user:  Maxim Dounin 
date:  Wed Sep 13 15:52:01 2017 +0300
description:
Fixed ngx_gmtime() on 32-bit platforms with 64-bit time_t.

In ngx_gmtime(), instead of casting to ngx_uint_t we now work with
time_t directly.  This allows using dates after 2038 on 32-bit platforms
which use 64-bit time_t, notably NetBSD and OpenBSD.

As the code is not able to work with negative time_t values, argument
is now set to 0 for negative values.  As a positive side effect, this
results in Epoch being used for such values instead of a date in distant
future.

diffstat:

 src/core/ngx_times.c |  18 ++
 1 files changed, 10 insertions(+), 8 deletions(-)

diffs (37 lines):

diff --git a/src/core/ngx_times.c b/src/core/ngx_times.c
--- a/src/core/ngx_times.c
+++ b/src/core/ngx_times.c
@@ -300,23 +300,25 @@ void
 ngx_gmtime(time_t t, ngx_tm_t *tp)
 {
 ngx_int_t   yday;
-ngx_uint_t  n, sec, min, hour, mday, mon, year, wday, days, leap;
+ngx_uint_t  sec, min, hour, mday, mon, year, wday, days, leap;
 
 /* the calculation is valid for positive time_t only */
 
-n = (ngx_uint_t) t;
+if (t < 0) {
+t = 0;
+}
 
-days = n / 86400;
+days = t / 86400;
+sec = t % 86400;
 
 /* January 1, 1970 was Thursday */
 
 wday = (4 + days) % 7;
 
-n %= 86400;
-hour = n / 3600;
-n %= 3600;
-min = n / 60;
-sec = n % 60;
+hour = sec / 3600;
+sec %= 3600;
+min = sec / 60;
+sec %= 60;
 
 /*
  * the algorithm based on Gauss' formula,
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel

Re: proxy cache lock responses always has 500ms delay

[sivasara]

Wow, it is a great workaround. If the upstream response times are contained
proxy_cache_lock_timeout, this should work perfectly.
Thank you for the help.

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?2,276344,276350#msg-276350

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: proxy cache lock responses always has 500ms delay

[Roman Arutyunyan]

On Wed, Sep 13, 2017 at 07:47:25AM -0400, sivasara wrote:
> Ah.. thanks for the reply. 
> 500ms seems too large. Is there any way to decrease this wait time?

Currently there's no way to change 500ms to a different value.  What you can do
is reduce proxy_cache_lock_timeout (5s by default) to make the requests which
wait longer proceed with uncached proxying.  In fact, once this timeout expires,
you will have the last chance to check if the resource is already unlocked.

-- 
Roman Arutyunyan
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: Secure Link Md5 Implementation

[anish10dec]

Any Update Please 
How to use two secret Keys for Secure Link Md5.

Primary to be used by application which is in production and secondary for
application build which has been  rolled out with changed secret key i.e.
secondary.
So that application should work in both scenario meanwhile till the all the
users update the application

Please help 
Inside location or server block

secure_link $arg_tok,$arg_e;
secure_link_md5 "primarysecret$arg_tok$arg_e";
secure_link_md5 "secondarysecret$arg_tok$arg_e";
if ($secure_link = "") {return 405;}
if ($secure_link = "0"){return 410;}

This gives error as secure link md5 is used twice within a location block

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?2,275668,276348#msg-276348

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

Re: proxy cache lock responses always has 500ms delay

[sivasara]

Ah.. thanks for the reply. 
500ms seems too large. Is there any way to decrease this wait time?

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?2,276344,276347#msg-276347

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

proxy cache lock responses always has 500ms delay

[sivasara]

Greetings everbody,

I have the following config. I give 3 simulatneous requests and 1 goes back
to the upstream and the 2 of them are in proxy_cache_lock. After the first
request completes, I am always seeing 500ms delay with proxy_cache_locked
requests. Is this expected behavior or am i missing something. 
Any help would be appreciated.

-nginx config:
user  nginx;
worker_processes  1;
error_log  logs/error.log;
pid/var/run/nginx.pid;
events {}
http {
proxy_cache_path /tmp/local_cache keys_zone=local_cache:250m levels=1:2
inactive=8s;
proxy_temp_path /tmp/proxy_temp;

log_format  main  '$remote_addr - $remote_user [$time_local] "$request"
'
  '$status $body_bytes_sent "$http_referer" '
  '"$http_user_agent" "$http_x_forwarded_for" '
  '$upstream_addr $upstream_response_time $request_time
$upstream_cache_status';

access_log  logs/access.log  main;
server {
listen   80;
server_name  localhost;

location / {
proxy_cache local_cache;
proxy_cache_valid 200 1m;
proxy_cache_lock on;
proxy_pass http://192.168.126.22:9095;
}

}
}

ab -n 4 -c 3 http://192.168.126.22/test
access.log:
192.168.126.22 - - [13/Sep/2017:15:14:36 +0530] "GET /test HTTP/1.0" 200 12
"-" "ApacheBench/2.3" "-" 192.168.126.22:9095 0.003 0.004 MISS
192.168.126.22 - - [13/Sep/2017:15:14:36 +0530] "GET /test HTTP/1.0" 200 12
"-" "ApacheBench/2.3" "-" - - 0.000 HIT
192.168.126.22 - - [13/Sep/2017:15:14:36 +0530] "GET /test HTTP/1.0" 200 12
"-" "ApacheBench/2.3" "-" - - 0.502 HIT
192.168.126.22 - - [13/Sep/2017:15:14:36 +0530] "GET /test HTTP/1.0" 200 12
"-" "ApacheBench/2.3" "-" - - 0.502 HIT

upstream:
curl http://192.168.126.22:9095/test
Hello world!

nginx:
nginx -V
nginx version: nginx/1.10.2
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC)
built with OpenSSL 1.0.1e-fips 11 Feb 2013
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx
--modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf
--error-log-path=/var/log/nginx/error.log
--http-log-path=/var/log/nginx/access.log
--http-client-body-temp-path=/var/lib/nginx/tmp/client_body
--http-proxy-temp-path=/var/lib/nginx/tmp/proxy
--http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi
--http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi
--http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid
--lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx
--with-file-aio --with-ipv6 --with-http_ssl_module --with-http_v2_module
--with-http_realip_module --with-http_addition_module
--with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic
--with-http_geoip_module=dynamic --with-http_sub_module
--with-http_dav_module --with-http_flv_module --with-http_mp4_module
--with-http_gunzip_module --with-http_gzip_static_module
--with-http_random_index_module --with-http_secure_link_module
--with-http_degradation_module --with-http_slice_module
--with-http_stub_status_module --with-http_perl_module=dynamic
--with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit
--with-stream=dynamic --with-stream_ssl_module
--with-google_perftools_module --with-debug --with-cc-opt='-O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong
--param=ssp-buffer-size=4 -grecord-gcc-switches
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic'
--with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
-Wl,-E'

Posted at Nginx Forum: 
https://forum.nginx.org/read.php?2,276344,276344#msg-276344

___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

AW: MP4 module with pseudo streaming + proxy_cache

[Lukas Tribus]

Hello,


> thanks for your comment Roman, do you know how these guys did it?
> https://www.maxcdn.com/one/tutorial/pseudo-streaming-maxcdn/

Why is pseudo streaming still a thing?

With HTML5 video players, everything is handled with RFC compliant
range requests and HTML5 video should be supported by pretty much
every browser in a stable release [1].

On linux, you need to install a codec like ffmpeg, but that still beats
flash.

I suggest you invest your time in HTML5 video and current technologies,
instead of the obsolete Adobe Flash with its URI based pseudo streaming.


cheers,
lukas


[1] https://en.wikipedia.org/wiki/HTML5_video#Browser_support
___
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx