HI Thanks for your great answer. you mentioned that sendfile() is to copy between kernel space and userland. I am curious, why this whole process don't need to malloc any memory? Could you please explain more on the detail implementation of the sendfile(). Many Thanks
Regards Muhui 2017-05-11 23:11 GMT+08:00 Maxim Dounin <[email protected]>: > Hello! > > On Thu, May 11, 2017 at 10:32:41PM +0800, Muhui Jiang wrote: > > > Recently, I did an experiment to test the memory consumption of nginx. I > > request a large static zip file. I explored the debug information of > nginx. > > > > For H2, below is a part of the log, I noticed that every time server will > > allocate 65536 bytes, I increase the connection number, I noticed that > the > > server's memory consumption will reach to a threshhold and then increase > > very slowly: > > [...] > > > 2017/05/11 04:54:20 [debug] 29451#0: *10499 http output filter > > "/image/test.zip?" > > 2017/05/11 04:54:20 [debug] 29451#0: *10499 http copy filter: > > "/image/test.zip?" > > 2017/05/11 04:54:20 [debug] 29451#0: *10499 malloc: > 0000000002699A80:65536 > > 2017/05/11 04:54:20 [debug] 29451#0: *10499 read: 14, 0000000002699A80, > > 65536, 0 > > [...] > > > 2017/05/11 04:54:20 [debug] 29451#0: *10499 http2 frame out: > > 00000000026155F0 sid:1 bl:0 len:1 > > 2017/05/11 04:54:20 [debug] 29451#0: *10499 SSL buf copy: 9 > > 2017/05/11 04:54:20 [debug] 29451#0: *10499 SSL buf copy: 1 > > 2017/05/11 04:54:20 [debug] 29451#0: *10499 SSL to write: 138 > > 2017/05/11 04:54:20 [debug] 29451#0: *10499 SSL_write: 138 > > 2017/05/11 04:54:20 [debug] 29451#0: *10499 http2:1 DATA frame > > 00000000026155F0 was sent > > [...] > > > For H/1.1, below is a part of the debug log, no malloc is noticed during > > the send file process. And even when I increase the connection number to > a > > very large value, the result shows nginx's memory consumption is still > very > > low. : > > [...] > > > 2017/05/11 22:29:06 [debug] 29451#0: *11015 http write filter limit 0 > > 2017/05/11 22:29:06 [debug] 29451#0: *11015 sendfile: @72470952 584002 > > 2017/05/11 22:29:06 [debug] 29451#0: *11015 sendfile: 260640 of 584002 > > [...] > > > Hope to get your comments and what are the difference of nginx's memory > > allocation mechanisms between HTTP/2.0 and HTTP/1.1. Many Thanks > > The difference is due to sendfile(), which is used in case of > plain HTTP, and can't be used with SSL-encrypted connections. > HTTP/2 is normally used with SSL encryption, so it is usually not > possible to use sendfile() with HTTP/2. > > When sendfile() is not available or switched off, nginx uses > output_buffers (http://nginx.org/r/output_buffers) to read a file > from disk, and then writes these buffers to the connection. > > When it is possible to use the sendfile(), nginx does not try to > read contents of static files it returns, but simply calls > sendfile(). This is usually most effecient approach , as it > avoids additional buffers and copying between kernel space and > userland. Unfortunately, it is not available when using HTTPS > (including HTTP/2). > > -- > Maxim Dounin > http://nginx.org/ > _______________________________________________ > nginx mailing list > [email protected] > http://mailman.nginx.org/mailman/listinfo/nginx >
_______________________________________________ nginx mailing list [email protected] http://mailman.nginx.org/mailman/listinfo/nginx
