Re: Module External Configuration
One last question about shared memory... I have my configuration now being loaded successfully into a shared memory segment.. now my problem is that someone could change the config so that the resulting structure wouldn't fit in the shared memory segment. Is it possible to in the child replace my current shared memory segment with a bigger one? I tried destroy()ing and then create()ing but that resulted in a segfault. Should it have worked? Is there a different way? Jason On Mon, Jun 20, 2011 at 5:20 PM, Nick Kew n...@apache.org wrote: On Mon, 20 Jun 2011 16:10:12 -0400 Mike Meyer m...@mired.org wrote: I assume that this is because a new process was spawned to handle a new request and the updated memory didn't get carried over (even though the pointer address didn't change...) A new process may be spawned from time to time (though not directly to handle a new request unless you have a very non-standard MPM). You could check config data in a child_init hook. However, if you have this problem, it's probably not your only one. The data reverting suggests that your module's config may be held on the server/config pool. If that's being updated (and if the updates involve any allocation at all) it's a memory leak. Your module should create its own config pool at child_init. Then it can read its own config data immediately, and - crucially - whenever it re-reads the data it can first clean that pool to reclaim memory. Put the configuration data into shared memory. Create and load the shared memory in the post config hook. Map the shared memory and potentially reload it during the child init hook. You'll need to use appropriate locking if you decide to reload it. Details on the locking will depend on the data structure details. If you use shared memory, note that 2.3/2.4 have much better support for it than earlier versions, with mod_socache and mod_slotmem. However, this may not be the best thing to do. You need to weigh up the cost of maintaining a per-process copy of the config data against that of using shared memory. Both are valid solutions. Also, do you want the overhead of a stat() every request to see if anything has changed, or could it work on a time basis, so it only performs a stat() if a certain time has elapsed since the last time? -- Nick Kew Available for work, contract or permanent. http://www.webthing.com/~nick/cv.html
Re: Module External Configuration
On Tue, 2011-06-21 at 16:26 -0500, Jason Funk wrote: One last question about shared memory... I have my configuration now being loaded successfully into a shared memory segment.. now my problem is that someone could change the config so that the resulting structure wouldn't fit in the shared memory segment. Is it possible to in the child replace my current shared memory segment with a bigger one? I tried destroy()ing and then create()ing but that resulted in a segfault. Should it have worked? Is there a different way? Jason segfaults are good indicators of pointers pointing to the wrong things. It will work if you do it right. Is your pointer to the shared memory in shared memory? Or do the clients have their own pointer that might change if the data in shm changes location? You might have to shm the pointer to a separate shm segment, and then after reading in a new config, change the shm pointer to point to the new one and destroy the old config. Don't touch the pointer aside from changing where it points to. Joe Lewis -- http://www.silverhawk.net/
Re: Module External Configuration
On Tue, Jun 21, 2011 at 23:26, Jason Funk jasonlf...@gmail.com wrote: One last question about shared memory... I have my configuration now being loaded successfully into a shared memory segment.. now my problem is that someone could change the config so that the resulting structure wouldn't fit in the shared memory segment. Is it possible to in the child replace my current shared memory segment with a bigger one? I tried destroy()ing and then create()ing but that resulted in a segfault. Should it have worked? Is there a different way? As I've said before, no, you cannot portably resize a shared memory segment. APR doesn't even expose that functionality. If you're targeting Unices only, you can use Sys V or POSIX IPC: you open() or shm_open() a memory segment, then ftruncate() it to the desired size. Make sure to wrap the call to ftruncate() in an exclusive lock or bad things will happen.
Re: Module External Configuration
On Tue, Jun 21, 2011 at 23:26, Jason Funk jasonlf...@gmail.com wrote: One last question about shared memory... I have my configuration now being loaded successfully into a shared memory segment.. now my problem is that someone could change the config so that the resulting structure wouldn't fit in the shared memory segment. Is it possible to in the child replace my current shared memory segment with a bigger one? I tried destroy()ing and then create()ing but that resulted in a segfault. Should it have worked? Is there a different way? Destroying the shared memory segment with the conf could be dangerous as other children could serve requests that are using the conf in the segment that you are destroying. Protecting with interprocess locks significantly slows down your server. Maybe a shared-exclusive lock is more efficient, I don't know. You could use a double buffer approach and reference counters. For example you load the new conf in buffer 2, then you update the pointer that points to the active conf, setting it to buffer 2. Then you check the reference counter of buffer 1 and when it reaches zero, you destroy buffer 1. The pointer that points to the active buffer, as well as the reference counters have to be mapped in shared memory too. S Jason On Mon, Jun 20, 2011 at 5:20 PM, Nick Kew n...@apache.org wrote: On Mon, 20 Jun 2011 16:10:12 -0400 Mike Meyer m...@mired.org wrote: I assume that this is because a new process was spawned to handle a new request and the updated memory didn't get carried over (even though the pointer address didn't change...) A new process may be spawned from time to time (though not directly to handle a new request unless you have a very non-standard MPM). You could check config data in a child_init hook. However, if you have this problem, it's probably not your only one. The data reverting suggests that your module's config may be held on the server/config pool. If that's being updated (and if the updates involve any allocation at all) it's a memory leak. Your module should create its own config pool at child_init. Then it can read its own config data immediately, and - crucially - whenever it re-reads the data it can first clean that pool to reclaim memory. Put the configuration data into shared memory. Create and load the shared memory in the post config hook. Map the shared memory and potentially reload it during the child init hook. You'll need to use appropriate locking if you decide to reload it. Details on the locking will depend on the data structure details. If you use shared memory, note that 2.3/2.4 have much better support for it than earlier versions, with mod_socache and mod_slotmem. However, this may not be the best thing to do. You need to weigh up the cost of maintaining a per-process copy of the config data against that of using shared memory. Both are valid solutions. Also, do you want the overhead of a stat() every request to see if anything has changed, or could it work on a time basis, so it only performs a stat() if a certain time has elapsed since the last time? -- Nick Kew Available for work, contract or permanent. http://www.webthing.com/~nick/cv.html
