Hello! On Tue, Jul 29, 2014 at 10:46:03PM +0200, Richard Stanway wrote:
> Hello, > I recently came across a modified version of zlib with code contributed by > Intel [1] that makes use of modern CPU instructions to increase > performance. In testing, the performance gains seemed substantial, however > when I tried to use this version with nginx, the following alert types > appeared in the error_log on gzip requests: > > 2014/07/30 05:20:29 [alert] 22285#0: *739837 gzip filter failed to use > preallocated memory: 65536 of 65520 while sending to client > 2014/07/30 05:40:42 [alert] 29462#0: *230460 gzip filter failed to use > preallocated memory: 1040 of 4112 > > It appears nginx is pre-allocating a buffer based on the original zlib > memory usage patterns, however the Intel optimized version has slightly > higher memory requirements due to padding for SSE functions etc. > > Is there a chance this version could be supported by nginx, or a > configuration option made available to control the allocation size? > > Thanks. > > [1] https://github.com/jtkukunas/zlib This version indeed uses larger allocations (much larger in some cases), and also uses different sizes which confuses nginx code when it tries to allocate 8k for deflate state. Quick hack to do correct preallocations for this zlib version: --- a/src/http/modules/ngx_http_gzip_filter_module.c Wed Jul 09 12:27:15 2014 -0700 +++ b/src/http/modules/ngx_http_gzip_filter_module.c Wed Jul 30 05:57:50 2014 +0400 @@ -521,7 +521,16 @@ ngx_http_gzip_filter_memory(ngx_http_req * *) 5920 bytes on amd64 and sparc64 */ +#if 0 ctx->allocated = 8192 + (1 << (wbits + 2)) + (1 << (memlevel + 9)); +#endif + + if (conf->level == 1) { + wbits = 13; + } + + ctx->allocated = 8192 + 16 + (1 << (wbits + 2)) + + (1 << (ngx_max(memlevel, 8) + 8)) + (1 << (memlevel + 8)); } @@ -987,7 +996,7 @@ ngx_http_gzip_filter_alloc(void *opaque, alloc = items * size; - if (alloc % 512 != 0 && alloc < 8192) { + if (alloc % 512 != 0 && alloc < 8192 && items == 1) { /* * The zlib deflate_state allocation, it takes about 6K, This is expected to work with normal zlib as well, but will be suboptimal from memory usage point of view. Note well that alerts logged aren't really serious - nginx is able to handle such cases properly, and will fallback to normal pool allocations instead of using preallocated block. It's just not something it expects to ever happen, so it logs alerts to make sure it's noticed. With different zlib libraries out there, we probably want to just silence the alert, keeping preallocation tuned for normal zlib. -- Maxim Dounin http://nginx.org/ _______________________________________________ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx