Re: Default log file locations
Hmm... >From _The Linux Programming Interface: A Linux and UNIX System Programming Handbook_ [2]: Nonblocking mode can be used with devices (e.g., terminals and pseudoterminals), pipes, FIFOs, and sockets. (Because file descriptors for pipes and sockets are not obtained using open(), we must enable this flag using the fcntl() F_SETFL operation described in Section 5.3.) So basically if a "sharing" of standard handles is properly implemented between master/workers and a line buffering is good enough for the logging, you can "write to terminal in non-blocking manner." Additionally note the stdout can be mapped to systemd journals (which are files and channel will be to pipe), if nginx would running as systemd service unit. Regards, Serg. 27.06.2019 19:35, Valentin V. Bartenev wrote: > Afaik, there's no way in Linux systems to write to terminal in > non-blocking manner. As the result, writing log can block the > whole nginx worker process and cause DoS. > > IMHO, it's not a good idea to make your web-application depend > on logging capabilities. > > wbr, Valentin V. Bartenev > > ___ > nginx-devel mailing list > nginx-devel@nginx.orghttp://mailman.nginx.org/mailman/listinfo/nginx-devel [1] Links: -- [1] http://mailman.nginx.org/mailman/listinfo/nginx-devel [2] https://books.google.de/books?id=Ps2SH727eCIClpg=PA103ots=kMHcB5EQyadq=From%20The%20Linux%20Programming%20Interface%3A%20A%20Linux%20and%20UNIX%20System%20Programming%20Handbook%3A%20Nonblocking%20mode%20can%20be%20used%20with%20devices%20(e.g.%2C%20terminals%20and%20pseudoterminals)%2C%20pipes%2C%20FIFOs%2C%20and%20sockets.%20(Because%20file%20descriptors%20for%20pipes%20and%20sockets%20are%20not%20obtained%20using%20open()%2C%20we%20must%20enable%20The%20Linux%20Programming%20Interface%3A%20A%20Linux%20and%20UNIX%20System%20Programming%20Handbookhl=depg=PA103#v=onepageqf=false___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
Re: Default log file locations
On Thursday 27 June 2019 19:44:22 Peter Bittner wrote: > I may be a bit ignorant about system-level operations, I apologize. > > Can you explain why this works better with writing to a file (i.e. the > hardcoded location /var/log/nginx/*.log)? > [..] In most cases writing to a file never blocks. Writing to stdout/stderr blocks as soon as buffers in kernel are getting full. This may happen when consumer is slower than producer, or if the consumer is just got stuck and don't read. You can see users experience this issue: https://github.com/docker/compose/issues/6018 So, nginx (like any other async application) must either log to a regular file or an UDP socket. Anything else is a bad idea from performance and reliability points of view. wbr, Valentin V. Bartenev ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
Re: Default log file locations
I may be a bit ignorant about system-level operations, I apologize. Can you explain why this works better with writing to a file (i.e. the hardcoded location /var/log/nginx/*.log)? Peter On Thu 27 June 2019 19:35 Valentin V. Bartenev wrote: > > On Thursday 27 June 2019 19:12:51 Peter Bittner wrote: > > Hi there! > > > > In my day job I'm helping to get applications from traditional > > environments running in cloud environments. Cloud native applications > > are just "normal" applications, but there are a few properties that > > they should satisfy (apart from resiliency and scalability). > > > > For logging this boils down to what is prescribed by the 12-factor > > app: The log output should be a continuous stream, i.e. simply log to > > the terminal. > > > > Now, as of today, at least on Debian based container images, the > > behavior of Nginx is to write to /var/log/nginx/access.log and > > /var/log/nginx/error.log by default. We try to compensate this by > > making those files symbolic links to /dev/stdout and /dev/stderr. > > We're doing this also, because there seem to be cases when a log entry > > is written _before_ it is configured via the Nginx configuration file. > > > > From my perspective it would be advantageous to have Nginx write to > > the terminal by default (i.e. no hardcoded log file locations) and > > allow to override this behavior via the Nginx configuration file. > > > > Is there any reason why the default behavior is not that way yet? > [..] > > Afaik, there's no way in Linux systems to write to terminal in > non-blocking manner. As the result, writing log can block the > whole nginx worker process and cause DoS. > > IMHO, it's not a good idea to make your web-application depend > on logging capabilities. > > wbr, Valentin V. Bartenev ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
Re: Default log file locations
On Thursday 27 June 2019 19:12:51 Peter Bittner wrote: > Hi there! > > In my day job I'm helping to get applications from traditional > environments running in cloud environments. Cloud native applications > are just "normal" applications, but there are a few properties that > they should satisfy (apart from resiliency and scalability). > > For logging this boils down to what is prescribed by the 12-factor > app: The log output should be a continuous stream, i.e. simply log to > the terminal. > > Now, as of today, at least on Debian based container images, the > behavior of Nginx is to write to /var/log/nginx/access.log and > /var/log/nginx/error.log by default. We try to compensate this by > making those files symbolic links to /dev/stdout and /dev/stderr. > We're doing this also, because there seem to be cases when a log entry > is written _before_ it is configured via the Nginx configuration file. > > From my perspective it would be advantageous to have Nginx write to > the terminal by default (i.e. no hardcoded log file locations) and > allow to override this behavior via the Nginx configuration file. > > Is there any reason why the default behavior is not that way yet? [..] Afaik, there's no way in Linux systems to write to terminal in non-blocking manner. As the result, writing log can block the whole nginx worker process and cause DoS. IMHO, it's not a good idea to make your web-application depend on logging capabilities. wbr, Valentin V. Bartenev ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
Default log file locations
Hi there! In my day job I'm helping to get applications from traditional environments running in cloud environments. Cloud native applications are just "normal" applications, but there are a few properties that they should satisfy (apart from resiliency and scalability). For logging this boils down to what is prescribed by the 12-factor app: The log output should be a continuous stream, i.e. simply log to the terminal. Now, as of today, at least on Debian based container images, the behavior of Nginx is to write to /var/log/nginx/access.log and /var/log/nginx/error.log by default. We try to compensate this by making those files symbolic links to /dev/stdout and /dev/stderr. We're doing this also, because there seem to be cases when a log entry is written _before_ it is configured via the Nginx configuration file. >From my perspective it would be advantageous to have Nginx write to the terminal by default (i.e. no hardcoded log file locations) and allow to override this behavior via the Nginx configuration file. Is there any reason why the default behavior is not that way yet? Peter ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
[njs] Version bump.
details: https://hg.nginx.org/njs/rev/0119e834a202 branches: changeset: 1018:0119e834a202 user: Valentin Bartenev date: Thu Jun 27 18:54:21 2019 +0300 description: Version bump. diffstat: njs/njs.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 63cd8e6679ba -r 0119e834a202 njs/njs.h --- a/njs/njs.h Tue Jun 25 14:44:55 2019 +0300 +++ b/njs/njs.h Thu Jun 27 18:54:21 2019 +0300 @@ -11,7 +11,7 @@ #include -#define NJS_VERSION "0.3.3" +#define NJS_VERSION "0.3.4" #include ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
[njs] Removed unused nxt_lvlhsh_ctx_t structure.
details: https://hg.nginx.org/njs/rev/40f26bb516a6
branches:
changeset: 1020:40f26bb516a6
user: Valentin Bartenev
date: Thu Jun 27 18:55:34 2019 +0300
description:
Removed unused nxt_lvlhsh_ctx_t structure.
diffstat:
nxt/nxt_lvlhsh.h | 10 --
1 files changed, 0 insertions(+), 10 deletions(-)
diffs (20 lines):
diff -r 9db5e400480c -r 40f26bb516a6 nxt/nxt_lvlhsh.h
--- a/nxt/nxt_lvlhsh.h Thu Jun 27 18:55:34 2019 +0300
+++ b/nxt/nxt_lvlhsh.h Thu Jun 27 18:55:34 2019 +0300
@@ -79,16 +79,6 @@ typedef struct {
typedef struct {
-nxt_lvlhsh_test_t test;
-nxt_lvlhsh_alloc_talloc;
-nxt_lvlhsh_free_t free;
-
-/* The maximum allowed aligned shift. */
-uint32_t max_shift;
-} nxt_lvlhsh_ctx_t;
-
-
-typedef struct {
void *slot;
} nxt_lvlhsh_t;
___
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel
[njs] Removed unused nalloc parameter from lvlhsh allocation functions.
details: https://hg.nginx.org/njs/rev/9db5e400480c
branches:
changeset: 1019:9db5e400480c
user: Valentin Bartenev
date: Thu Jun 27 18:55:34 2019 +0300
description:
Removed unused nalloc parameter from lvlhsh allocation functions.
diffstat:
njs/njs_event.c | 1 -
njs/njs_extern.c| 2 --
njs/njs_lexer_keyword.c | 1 -
njs/njs_module.c| 1 -
njs/njs_object.c| 1 -
njs/njs_shell.c | 5 ++---
njs/njs_string.c| 1 -
njs/njs_variable.c | 2 --
njs/njs_vm.c| 2 +-
njs/njs_vm.h| 2 +-
njs/test/njs_unit_test.c| 3 +--
nxt/nxt_lvlhsh.c| 5 ++---
nxt/nxt_lvlhsh.h| 6 +-
nxt/test/lvlhsh_unit_test.c | 3 +--
14 files changed, 9 insertions(+), 26 deletions(-)
diffs (252 lines):
diff -r 0119e834a202 -r 9db5e400480c njs/njs_event.c
--- a/njs/njs_event.c Thu Jun 27 18:54:21 2019 +0300
+++ b/njs/njs_event.c Thu Jun 27 18:55:34 2019 +0300
@@ -15,7 +15,6 @@ const nxt_lvlhsh_proto_t njs_event_hash
nxt_aligned(64) =
{
NXT_LVLHSH_DEFAULT,
-0,
njs_event_hash_test,
njs_lvlhsh_alloc,
njs_lvlhsh_free,
diff -r 0119e834a202 -r 9db5e400480c njs/njs_extern.c
--- a/njs/njs_extern.c Thu Jun 27 18:54:21 2019 +0300
+++ b/njs/njs_extern.c Thu Jun 27 18:55:34 2019 +0300
@@ -50,7 +50,6 @@ const nxt_lvlhsh_proto_t njs_extern_has
nxt_aligned(64) =
{
NXT_LVLHSH_DEFAULT,
-NXT_LVLHSH_BATCH_ALLOC,
njs_extern_hash_test,
njs_lvlhsh_alloc,
njs_lvlhsh_free,
@@ -61,7 +60,6 @@ const nxt_lvlhsh_proto_t njs_extern_val
nxt_aligned(64) =
{
NXT_LVLHSH_DEFAULT,
-NXT_LVLHSH_BATCH_ALLOC,
njs_extern_value_hash_test,
njs_lvlhsh_alloc,
njs_lvlhsh_free,
diff -r 0119e834a202 -r 9db5e400480c njs/njs_lexer_keyword.c
--- a/njs/njs_lexer_keyword.c Thu Jun 27 18:54:21 2019 +0300
+++ b/njs/njs_lexer_keyword.c Thu Jun 27 18:55:34 2019 +0300
@@ -138,7 +138,6 @@ const nxt_lvlhsh_proto_t njs_keyword_ha
nxt_aligned(64) =
{
NXT_LVLHSH_DEFAULT,
-0,
njs_keyword_hash_test,
njs_lvlhsh_alloc,
njs_lvlhsh_free,
diff -r 0119e834a202 -r 9db5e400480c njs/njs_module.c
--- a/njs/njs_module.c Thu Jun 27 18:54:21 2019 +0300
+++ b/njs/njs_module.c Thu Jun 27 18:55:34 2019 +0300
@@ -415,7 +415,6 @@ const nxt_lvlhsh_proto_t njs_modules_ha
nxt_aligned(64) =
{
NXT_LVLHSH_DEFAULT,
-0,
njs_module_hash_test,
njs_lvlhsh_alloc,
njs_lvlhsh_free,
diff -r 0119e834a202 -r 9db5e400480c njs/njs_object.c
--- a/njs/njs_object.c Thu Jun 27 18:54:21 2019 +0300
+++ b/njs/njs_object.c Thu Jun 27 18:55:34 2019 +0300
@@ -149,7 +149,6 @@ const nxt_lvlhsh_proto_t njs_object_has
nxt_aligned(64) =
{
NXT_LVLHSH_DEFAULT,
-0,
njs_object_hash_test,
njs_lvlhsh_alloc,
njs_lvlhsh_free,
diff -r 0119e834a202 -r 9db5e400480c njs/njs_shell.c
--- a/njs/njs_shell.c Thu Jun 27 18:54:21 2019 +0300
+++ b/njs/njs_shell.c Thu Jun 27 18:55:34 2019 +0300
@@ -107,7 +107,7 @@ static void njs_console_clear_timer(njs_
njs_host_event_t event);
static nxt_int_t lvlhsh_key_test(nxt_lvlhsh_query_t *lhq, void *data);
-static void *lvlhsh_pool_alloc(void *pool, size_t size, nxt_uint_t nalloc);
+static void *lvlhsh_pool_alloc(void *pool, size_t size);
static void lvlhsh_pool_free(void *pool, void *p, size_t size);
@@ -192,7 +192,6 @@ static njs_external_t njs_externals[] =
static const nxt_lvlhsh_proto_t lvlhsh_proto nxt_aligned(64) = {
NXT_LVLHSH_LARGE_SLAB,
-0,
lvlhsh_key_test,
lvlhsh_pool_alloc,
lvlhsh_pool_free,
@@ -1227,7 +1226,7 @@ lvlhsh_key_test(nxt_lvlhsh_query_t *lhq,
static void *
-lvlhsh_pool_alloc(void *pool, size_t size, nxt_uint_t nalloc)
+lvlhsh_pool_alloc(void *pool, size_t size)
{
return nxt_mp_align(pool, size, size);
}
diff -r 0119e834a202 -r 9db5e400480c njs/njs_string.c
--- a/njs/njs_string.c Thu Jun 27 18:54:21 2019 +0300
+++ b/njs/njs_string.c Thu Jun 27 18:55:34 2019 +0300
@@ -4591,7 +4591,6 @@ static const nxt_lvlhsh_proto_t njs_val
nxt_aligned(64) =
{
NXT_LVLHSH_DEFAULT,
-0,
njs_values_hash_test,
njs_lvlhsh_alloc,
njs_lvlhsh_free,
diff -r 0119e834a202 -r 9db5e400480c njs/njs_variable.c
--- a/njs/njs_variable.cThu Jun 27 18:54:21 2019 +0300
+++ b/njs/njs_variable.cThu Jun 27 18:55:34 2019 +0300
@@ -37,7 +37,6 @@ const nxt_lvlhsh_proto_t njs_variables_
nxt_aligned(64) =
{
NXT_LVLHSH_DEFAULT,
-0,
njs_variables_hash_test,
njs_lvlhsh_alloc,
njs_lvlhsh_free,
@@ -232,7 +231,6 @@ const nxt_lvlhsh_proto_t njs_references
nxt_aligned(64) =
{
NXT_LVLHSH_DEFAULT,
-0,
njs_reference_hash_test,
njs_lvlhsh_alloc,
njs_lvlhsh_free,
diff -r 0119e834a202 -r 9db5e400480c njs/njs_vm.c
--- a/njs/njs_vm.c Thu Jun 27 18:54:21 2019 +0300
+++ b/njs/njs_vm.c Thu Jun 27 18:55:34 2019 +0300
@@ -3672,7
