Re: internal redirect and module context
That is exactly what I needed. Thank you. Dk. On Sat, Jan 22, 2022 at 11:08 AM Maxim Dounin wrote: > Hello! > > On Sat, Jan 22, 2022 at 02:19:05AM -0800, Dk Jack wrote: > > > Maxim, > > Thanks for responding to my query. I am passing the original context > > pointer to the clean up handler. When my cleanup handler is called I am > > retrieving the context pointer to clean up external resources. Based on > > your response, the pointer saved in the cleanup handler should still be > > valid and should be still safe to use and no memory/resources will be > > leaked if I use that pointer to cleanup old allocations. This seems to be > > in agreement with what I observed in my debugging. > > Yes, that's look correct. Note that if you create a new > context with new resources, you have to add another cleanup > handler to free these new resources as well. > > > A follow up question. After the redirect call, I am recreating the > context > > and restoring some of the data. However, like you mentioned I cannot > > restore all the data. Currently, I am not accessing inaccessible data, > > it seems to be working fine. However, in case I need to access the lost > > data, is there another area in the request that is not disturbed by the > > redirect call where I can save the context data? > > Cleanup handlers is the best way go, check the realip module and > the ngx_http_realip_get_module_ctx() function I've mentioned in > the previous message. > > -- > Maxim Dounin > http://mdounin.ru/ > ___ > nginx-devel mailing list -- nginx-devel@nginx.org > To unsubscribe send an email to nginx-devel-le...@nginx.org > ___ nginx-devel mailing list -- nginx-devel@nginx.org To unsubscribe send an email to nginx-devel-le...@nginx.org
Re: internal redirect and module context
Hello! On Sat, Jan 22, 2022 at 02:19:05AM -0800, Dk Jack wrote: > Maxim, > Thanks for responding to my query. I am passing the original context > pointer to the clean up handler. When my cleanup handler is called I am > retrieving the context pointer to clean up external resources. Based on > your response, the pointer saved in the cleanup handler should still be > valid and should be still safe to use and no memory/resources will be > leaked if I use that pointer to cleanup old allocations. This seems to be > in agreement with what I observed in my debugging. Yes, that's look correct. Note that if you create a new context with new resources, you have to add another cleanup handler to free these new resources as well. > A follow up question. After the redirect call, I am recreating the context > and restoring some of the data. However, like you mentioned I cannot > restore all the data. Currently, I am not accessing inaccessible data, > it seems to be working fine. However, in case I need to access the lost > data, is there another area in the request that is not disturbed by the > redirect call where I can save the context data? Cleanup handlers is the best way go, check the realip module and the ngx_http_realip_get_module_ctx() function I've mentioned in the previous message. -- Maxim Dounin http://mdounin.ru/ ___ nginx-devel mailing list -- nginx-devel@nginx.org To unsubscribe send an email to nginx-devel-le...@nginx.org
Re: internal redirect and module context
Maxim, Thanks for responding to my query. I am passing the original context pointer to the clean up handler. When my cleanup handler is called I am retrieving the context pointer to clean up external resources. Based on your response, the pointer saved in the cleanup handler should still be valid and should be still safe to use and no memory/resources will be leaked if I use that pointer to cleanup old allocations. This seems to be in agreement with what I observed in my debugging. A follow up question. After the redirect call, I am recreating the context and restoring some of the data. However, like you mentioned I cannot restore all the data. Currently, I am not accessing inaccessible data, it seems to be working fine. However, in case I need to access the lost data, is there another area in the request that is not disturbed by the redirect call where I can save the context data? Regards, Dk. - By saying "all request contexts are erased" the development guide means that module contexts will no longer be available via the ngx_http_get_module_ctx() macro. The ngx_http_internal_redirect() clears the r->ctx[] array of pointers: /* clear the modules contexts */ ngx_memzero(r->ctx, sizeof(void *) * ngx_http_max_module); That is, the actual memory you've used for your module context will be intact, but the pointer you've saved into request with ngx_http_set_module_ctx() macro will no longer be returned by the ngx_http_get_module_ctx() macro. Usually, modules will simply re-allocate context as needed if it's not present. As long as context memory is allocated from the request pool, this won't cause a memory leak: all memory allocated from the request pool is automatically released when the request pool is destroyed. If you are allocating some external resources, such as open file descriptors or memory allocated directly from OS, you'll have to use cleanup handler free these resources. In this case, you have to make sure that the cleanup handler you've installed will free all the external resources you've allocated. Usually this means that you'll have to add a cleanup handler per resource: for example, nginx adds a cleanup handler for each file it opens, see ngx_open_cached_file(). Or, if you are keeping pointers to the allocated resources in your module context, a cleanup handler per each allocated context might be a good option. Note well that in some cases it might not possible to re-create module context, for example, if some information is no longer available. In this case it is possible to preserve the module context by saving the pointer elsewhere, and restoring it if possible instead of re-allocating if ngx_http_get_module_ctx() returns NULL. For example, the realip module uses request pool cleanup handlers to save and restore its context when needed, see ngx_http_realip_get_module_ctx(). Hope this helps. -- Maxim Dounin http://mdounin.ru/ ___ nginx-devel mailing list -- nginx-devel@nginx.org To unsubscribe send an email to nginx-devel-le...@nginx.org On Fri, Jan 21, 2022 at 4:07 PM Dk Jack wrote: > Hi, > I have a question related to internal redirect, I am hoping someone from > this forum can clarify. The email is a bit long since I wanted to provide > enough background for my situation. > > In my module, I am creating my module context and saving some state. Some > of this state is allocated using ngx_palloc. I am releasing this memory by > adding a pool clean up handler. > > In my module, for certain requests I am doing an internal redirect. My > code for redirect looks something like this: > > ngx_http_internal_redirect(r, _uri, >args); > ngx_http_finalize_request(r, NGX_DONE); > > According to the documentation > > http://nginx.org/en/docs/dev/development_guide.html#http_request_redirection > > it says, on calling internal_redirect, the module context will be erased > to avoid inconsistencies. It also says, the processing returns to the > NGX_HTTP_SERVER_REWRITE_PHASE. To understand the behavior better, I > attached a debugger and added a breakpoint after the above two lines. When > the debugger stopped at my breakpoint, my module context still seems to be > valid. I added a second breakpoint in my rewrite-handler and allowed the > debugger to continue. Now when the debugger stopped at the second > breakpoint, my module context was erased which seems consistent with the > documentation. > > So my question is, if my context is erased, what happens to the memory I > allocated before my module invoked the internal redirect call? I put a log > statement in my cleanup function and I observed that it is getting invoked > only once at the end of the request processing. It is not getting called > when my context is erased after an
Re: internal redirect and module context
Hello! On Fri, Jan 21, 2022 at 04:07:26PM -0800, Dk Jack wrote: > I have a question related to internal redirect, I am hoping someone from > this forum can clarify. The email is a bit long since I wanted to provide > enough background for my situation. > > In my module, I am creating my module context and saving some state. Some > of this state is allocated using ngx_palloc. I am releasing this memory by > adding a pool clean up handler. > > In my module, for certain requests I am doing an internal redirect. My code > for redirect looks something like this: > > ngx_http_internal_redirect(r, _uri, >args); > ngx_http_finalize_request(r, NGX_DONE); > > According to the documentation > http://nginx.org/en/docs/dev/development_guide.html#http_request_redirection > > it says, on calling internal_redirect, the module context will be erased to > avoid inconsistencies. It also says, the processing returns to the > NGX_HTTP_SERVER_REWRITE_PHASE. To understand the behavior better, I > attached a debugger and added a breakpoint after the above two lines. When > the debugger stopped at my breakpoint, my module context still seems to be > valid. I added a second breakpoint in my rewrite-handler and allowed the > debugger to continue. Now when the debugger stopped at the second > breakpoint, my module context was erased which seems consistent with the > documentation. > > So my question is, if my context is erased, what happens to the memory I > allocated before my module invoked the internal redirect call? I put a log > statement in my cleanup function and I observed that it is getting invoked > only once at the end of the request processing. It is not getting called > when my context is erased after an internal redirect. Since I need my > context data after redirection, do I reallocate and recreate it? Since my > clean up code is getting called only once. Would this lead to a memory leak > if I reallocated after the internal redirect call because I'd be allocated > once before redirect and once after redirect. Any help or clarification in > this regard is greatly appreciated. By saying "all request contexts are erased" the development guide means that module contexts will no longer be available via the ngx_http_get_module_ctx() macro. The ngx_http_internal_redirect() clears the r->ctx[] array of pointers: /* clear the modules contexts */ ngx_memzero(r->ctx, sizeof(void *) * ngx_http_max_module); That is, the actual memory you've used for your module context will be intact, but the pointer you've saved into request with ngx_http_set_module_ctx() macro will no longer be returned by the ngx_http_get_module_ctx() macro. Usually, modules will simply re-allocate context as needed if it's not present. As long as context memory is allocated from the request pool, this won't cause a memory leak: all memory allocated from the request pool is automatically released when the request pool is destroyed. If you are allocating some external resources, such as open file descriptors or memory allocated directly from OS, you'll have to use cleanup handler free these resources. In this case, you have to make sure that the cleanup handler you've installed will free all the external resources you've allocated. Usually this means that you'll have to add a cleanup handler per resource: for example, nginx adds a cleanup handler for each file it opens, see ngx_open_cached_file(). Or, if you are keeping pointers to the allocated resources in your module context, a cleanup handler per each allocated context might be a good option. Note well that in some cases it might not possible to re-create module context, for example, if some information is no longer available. In this case it is possible to preserve the module context by saving the pointer elsewhere, and restoring it if possible instead of re-allocating if ngx_http_get_module_ctx() returns NULL. For example, the realip module uses request pool cleanup handlers to save and restore its context when needed, see ngx_http_realip_get_module_ctx(). Hope this helps. -- Maxim Dounin http://mdounin.ru/ ___ nginx-devel mailing list -- nginx-devel@nginx.org To unsubscribe send an email to nginx-devel-le...@nginx.org
internal redirect and module context
Hi, I have a question related to internal redirect, I am hoping someone from this forum can clarify. The email is a bit long since I wanted to provide enough background for my situation. In my module, I am creating my module context and saving some state. Some of this state is allocated using ngx_palloc. I am releasing this memory by adding a pool clean up handler. In my module, for certain requests I am doing an internal redirect. My code for redirect looks something like this: ngx_http_internal_redirect(r, _uri, >args); ngx_http_finalize_request(r, NGX_DONE); According to the documentation http://nginx.org/en/docs/dev/development_guide.html#http_request_redirection it says, on calling internal_redirect, the module context will be erased to avoid inconsistencies. It also says, the processing returns to the NGX_HTTP_SERVER_REWRITE_PHASE. To understand the behavior better, I attached a debugger and added a breakpoint after the above two lines. When the debugger stopped at my breakpoint, my module context still seems to be valid. I added a second breakpoint in my rewrite-handler and allowed the debugger to continue. Now when the debugger stopped at the second breakpoint, my module context was erased which seems consistent with the documentation. So my question is, if my context is erased, what happens to the memory I allocated before my module invoked the internal redirect call? I put a log statement in my cleanup function and I observed that it is getting invoked only once at the end of the request processing. It is not getting called when my context is erased after an internal redirect. Since I need my context data after redirection, do I reallocate and recreate it? Since my clean up code is getting called only once. Would this lead to a memory leak if I reallocated after the internal redirect call because I'd be allocated once before redirect and once after redirect. Any help or clarification in this regard is greatly appreciated. Dk. ___ nginx-devel mailing list -- nginx-devel@nginx.org To unsubscribe send an email to nginx-devel-le...@nginx.org
module context.
Hi, I have a module. I am saving some data in my module context. However, in certain cases, I am doing an internal redirect. When I do an internal redirect my context is cleared as per the nginx development guide. Is there a way to save the context data. Since this data is specific to the request, I want to save it in the ngx_http_request_t if possible. My current solution is to save it as header values and retrieve them later after the internal redirect has happened. I am setting a special header to say, I've saved data. This is not an elegant solution but it works. The problem is, since I am saving it in headers, the header I added to save my context data is being sent to the origin as a header. After restoring my context, I tried setting the header key and value lengths to zero and also tried setting header hash to zero. None of these things seem to help and the header is still getting transmitted. Two questions: Is there a way to save module context data in ngx_http_request_t when an internal redirect happens? If not, is there a way to prevent the custom headers add by module from going out? Thanks for help. Dk. ___ nginx-devel mailing list nginx-devel@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel
