Re: [PHP-DEV] Re: PHP-FPM process management woes
On Thu, Dec 23, 2021 at 5:32 AM Christoph M. Becker wrote: > > On 22.12.2021 at 22:44, Jakub Zelenka wrote: > > > After thinking about this a bit more I think we would still need some way > > to provide the current functionality of the MINIT if that's moved on child > > level. The problem with not having such step is that opcache shm would be > > then segmented between children - each child process would have its own shm > > so it means that it would likely break things like opcache_reset that would > > work only for a single child but wouldn't have any impact on other > > children. > > I think the children could still re-attach to OPcache (like on Windows), > and that might not even be that bad as on Windows (where ASLR can break > that). However, forking late might break preloading (not sure), and > generally re-attaching isn't the nicest solution. I remember some similar issues a while back. I wonder if it could make sense to separate these two flows. The MINIT is critical to many extensions, not doing fancy things like OpCache or similar. Would it be possible to split them? One actual MINIT (as per doc, once per process) and one for more fancy stuff like what is done in OpCache (or apcu afair)? I can't think of another way to make the MINIT/SHUTDOWN API behave as it should while being designed before fpm came to life. Or we let the ext developers handle it themselves by providing some helpers function about the current stage the ext is in (root process or childs) but that will be painful to do and port. Best, -- Pierre @pierrejoye | http://www.libgd.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
[PHP-DEV] Re: PHP-FPM process management woes
On 22.12.2021 at 22:44, Jakub Zelenka wrote: > After thinking about this a bit more I think we would still need some way > to provide the current functionality of the MINIT if that's moved on child > level. The problem with not having such step is that opcache shm would be > then segmented between children - each child process would have its own shm > so it means that it would likely break things like opcache_reset that would > work only for a single child but wouldn't have any impact on other > children. I think the children could still re-attach to OPcache (like on Windows), and that might not even be that bad as on Windows (where ASLR can break that). However, forking late might break preloading (not sure), and generally re-attaching isn't the nicest solution. Christoph -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
[PHP-DEV] Re: PHP-FPM process management woes
> >> My suggestion for a fix would be to emulate what Apache always did: >> >> One MINIT/MSHUTDOWN in the main control process (I think it needed that >> to be able to implement php_admin_value and php_value), and then >> additionally also in each worker process. >> > > As I said above, it won't probably fully fix your problem but if you still > want to try to tackle it and move the MINIT, the way that I would do it is > to try to separate the whole sapi init logic and call it from the child > init as the first thing. If you want to experiment with using > php_admin_value before the module minit, then it might be worth to try put > fpm_php_init_child (or just the defines) to sapi startup callback before > calling php_module_startup so the main INIs are loads but it might be a bit > tricky to get worker pool config - it might need some extra globals maybe > for it. But not sure if that's gonna work (e.g. if the main INI files are > loaded at the sapi startup stage) and what will be impact on extensions. > I'd probably have to check it more and try it. Think it's also something > that could happen in master branch only as it's changing some fundamental > bits in FPM... > > After thinking about this a bit more I think we would still need some way to provide the current functionality of the MINIT if that's moved on child level. The problem with not having such step is that opcache shm would be then segmented between children - each child process would have its own shm so it means that it would likely break things like opcache_reset that would work only for a single child but wouldn't have any impact on other children. Also apcu would be segmented and basically anything that uses shm would be in some way impacted. The question is if maybe that single MINIT is actually what's optimal for most extension and that symmetry is just for some special cases which is actually my feeling. In such case it would be probably better to keep MINIT as it is and introduce new step for that symmetry with MSHUTDOWN. By the way that shm sharing on master level is not ideal because this should really be done on the pool level. I have been thinking about something called pool manager (process under master that would manage pool processes in the same or similar way as master does right know) for some time. That would allow this kind of shm sharing except other things. Regards Jakub
Re: [PHP-DEV] Re: PHP-FPM process management woes
As Jakub mentioned, exactly one MINIT/MSHUTDOWN per process will only fix one of your problems, but I do like the step in that direction. I express my sympathy and condolences: I've had my own share of woes with the fact that env var and INI changes are not complete during MINIT due to fcgi protocol/php-fpm. It's very annoying for extensions! I always install all hooks that might be needed, and have runtime checks if they are actually enabled. Not ideal, and sadly one MINIT/MSHUTDOWN will not fix it. -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: https://www.php.net/unsub.php
[PHP-DEV] Re: PHP-FPM process management woes
Hi, On Mon, Dec 20, 2021 at 6:41 PM Derick Rethans wrote: > Literature since at least 2006 (Sara's Extending and Embedding PHP > book), as well as numerous presentations that I have seen and given, and > including the online PHP Internals Book > ( > https://www.phpinternalsbook.com/php7/extensions_design/php_lifecycle.html), > > always expected that the MINIT (and MSHUTDOWN) functions are called once > per worker process. However, PHP-FPM does not do that. It only calls > extension's MINITs once in the main control process, but it *does* call > an MSHUTDOWN for each worker process. > In the past, this has already caused an issue where I couldn't create a > monitoring thread in MINIT, and then wait for it to end in MSHUTDOWN, as > MSHUTDOWN would wait on the same thread in each worker process, although > only one was created in MINIT. > > The way how PHP-FPM handles this breaks the generally assumed "one > MINIT, one MSHUTDOWN call" per process approach. > Yeah this is a bit unfortunate asymetry. It has got some slight advantages like in case of preloading that is done just once instead of on each child init. However there are most likely more disadvantages like the ones above and also the fact that MINIT is often run under the root so it's not ideal from the security point of view - especially when running 3rd party extensions. > In particular, in the case of Xdebug bug #2051 it creates a problem with > the following set-up: > > 1. php.ini has a `xdebug.mode=off` > 2. the pool configuration has a `php_admin_value[xdebug.mode] = debug` >directive > > So this is actually mainly about the fact that fpm_php_apply_defines runs after the MINIT. You might be able to tweak this but not sure if it completely fixes the issue as there's another to way overwrite INI that happens during the request. It is using FCGI env PHP_ADMIN_VALUE (see fastcgi_ini_parser usage) so you might need to handle this case in your code anyway. Basically you can't rely on the fact that INI stays the same so you will probably have to add some logic to your RINIT to handle this. > > My suggestion for a fix would be to emulate what Apache always did: > > One MINIT/MSHUTDOWN in the main control process (I think it needed that > to be able to implement php_admin_value and php_value), and then > additionally also in each worker process. > As I said above, it won't probably fully fix your problem but if you still want to try to tackle it and move the MINIT, the way that I would do it is to try to separate the whole sapi init logic and call it from the child init as the first thing. If you want to experiment with using php_admin_value before the module minit, then it might be worth to try put fpm_php_init_child (or just the defines) to sapi startup callback before calling php_module_startup so the main INIs are loads but it might be a bit tricky to get worker pool config - it might need some extra globals maybe for it. But not sure if that's gonna work (e.g. if the main INI files are loaded at the sapi startup stage) and what will be impact on extensions. I'd probably have to check it more and try it. Think it's also something that could happen in master branch only as it's changing some fundamental bits in FPM... Regards Jakub
