Thankyou for clarifying. RajKumar
On Wednesday 31 July 2024 at 11:56:48 UTC+5:30 Graham Dumpleton wrote: > If you define a daemon process group, the number of processes defined for > the group will always be started and running. > > With the way you have configured things your Python web application code > is only loaded lazily by a process in a daemon process group when the first > request arrives which is to be handled by that process. > > Thus, prior to any request being received, the memory footprint of the > processes should be similar to that of running an empty Python interpreter. > > If your web applications are infrequently accessed and you want to > minimise memory usage, then add to WSGIDaemonProcess the option: > > inactivity-timeout=30 > > See > https://modwsgi.readthedocs.io/en/master/configuration-directives/WSGIDaemonProcess.html > for > details, but basically what it means is that the process will be restarted > when idle and will revert memory usage by the process back to the base > level. > > Do note however that if you Python web application takes a long time to > load, then this may be noticeable to users if the Python web application > code is going to have to be reloaded frequently. > > As to CPU usage, the process if not handling requests will be waiting on > the socket on which requests are listening, so it should not be using any > CPU at that time. > > There are other options to WSGIDaemonProcess you could play with to > recycle the process periodically, but what might be appropriate really > depends on your specific web application so I can't give concrete > suggestions of what to use. > > Graham > > On 31 Jul 2024, at 2:55 PM, RajKumar Ambadipelli <arkki...@gmail.com> > wrote: > > What exactly I want is I want to know WSGIDaemonProcess which are ideal > not receiving any requests and not serving response but only declared, How > does this WSGIDaemonProcess effect my resources like memory and cpu. > > On Wednesday 31 July 2024 at 10:22:23 UTC+5:30 RajKumar Ambadipelli wrote: > >> Creating multiple virtualhost with different mod_wsgi daemons I meant >> 'WSGIDaemonProcess' >> and not using it that is it will not server and requests but only >> declared does this effect my resources like memory and CPU. >> On Monday 29 July 2024 at 12:45:21 UTC+5:30 Graham Dumpleton wrote: >> >>> Comes down to how much memory and CPU your machine has, plus how much >>> traffic the sites get. So will depend and you will just have to see how it >>> goes. >>> >>> On 29 Jul 2024, at 5:12 PM, RajKumar Ambadipelli <arkki...@gmail.com> >>> wrote: >>> >>> Yeah it is working fine what I did was >>> >>> step1. Unlinked existing symlink >>> Step2. Create new symlink >>> Step3. Managed permissions and shell context of files >>> >>> Therefore no >>> reload or restart is needed. >>> >>> But I have doubt up to how many mod_wsgi daemons we can efficiently >>> without any disturbance and up to what extent we can trust this deployment >>> process using mod_wsgi in daemon mode for production environment. >>> >>> Thankyou, >>> RajKumar >>> On Thursday 25 July 2024 at 13:04:18 UTC+5:30 RajKumar Ambadipelli wrote: >>> >>>> Sorry that uoadmin is actually admin. >>>> >>>> Ok I will try this one. >>>> >>>> Thankyou, >>>> RajKumar >>>> >>>> On Thursday 25 July 2024 at 12:59:13 UTC+5:30 Graham Dumpleton wrote: >>>> >>>>> You have two different directories which may complicate it a little, >>>>> but what you can do is have something like: >>>>> >>>>> Listen 9002 <VirtualHost *:9002> >>>>> ServerName test.myapp.com >>>>> ErrorLog /var/log/webservice_error.log >>>>> WSGIPassAuthorization On >>>>> WSGIDaemonProcess Tes9002 >>>>> python-path=/home/uoadmin/releases/test/students:/home/admin/releases/test/shared >>>>> >>>>> display-name=%{GROUP} >>>>> WSGIProcessGroup Tes9002 WSGIApplicationGroup %{GLOBAL} >>>>> WSGIScriptAlias / /home/admin/releases/test/students/conf/wsgi.py >>>>> <Directory /home/admin/releases/test/students/conf> >>>>> <Files wsgi.py> Require all granted </Files> >>>>> </Directory> >>>>> </VirtualHost> >>>>> >>>>> <VirtualHost *:9002> >>>>> ServerName dev.myapp.com >>>>> ErrorLog /var/log/webservice_error.log >>>>> WSGIPassAuthorization On >>>>> WSGIDaemonProcess Dev9002 >>>>> python-path=/home/uoadmin/releases/dev/students:/home/admin/releases/dev/shared >>>>> >>>>> display-name=%{GROUP} >>>>> WSGIProcessGroup Dev9002 WSGIApplicationGroup %{GLOBAL} >>>>> WSGIScriptAlias / /home/admin/releases/dev/students/conf/wsgi.py >>>>> <Directory /home/admin/releases/dev/students/conf> >>>>> <Files wsgi.py> Require all granted </Files> >>>>> </Directory> >>>>> </VirtualHost> >>>>> >>>>> In this case /home/admin/releases/dev would actually be a symlink to >>>>> the versioned directory. >>>>> >>>>> Not sure why have separate uoadmin directory, but also have >>>>> /home/uoadmin/releases/dev as symlink to the versioned directory. >>>>> >>>>> To change what is used, remove/recreate symlinks so point to the >>>>> directory for the new version. >>>>> >>>>> That the wsgi.py has changed should trigger a process restart if auto >>>>> reloading is on (default). >>>>> >>>>> Alternatively could disable auto reloading and manually send SIGUSR1 >>>>> to process to trigger a graceful reload after changing the symlink. >>>>> >>>>> That all said, I can't remember if you might have to configure Apache >>>>> to tell it to allow following symlinks (Options FollowSymLinks). If >>>>> it were static file handling would definitely be needed, but can't >>>>> remember >>>>> if would be required in this case where is WSGI application handling >>>>> things. >>>>> >>>>> Graham >>>>> >>>>> On 25 Jul 2024, at 5:21 PM, RajKumar Ambadipelli <arkki...@gmail.com> >>>>> wrote: >>>>> >>>>> Ok, If I have only two virtualhosts all the time and I am going to >>>>> change only python-path will that work with the direct signal using >>>>> SIGUSR1. >>>>> >>>>> If the above is possible then I think I have some kind of solution. >>>>> >>>>> Thankyou, >>>>> RajKumar >>>>> >>>>> On Thursday 25 July 2024 at 12:35:09 UTC+5:30 Graham Dumpleton wrote: >>>>> >>>>>> Are you always using the same two virtual host server names and just >>>>>> updating the version number in the paths? >>>>>> >>>>>> On 25 Jul 2024, at 4:21 PM, RajKumar Ambadipelli <arkki...@gmail.com> >>>>>> wrote: >>>>>> >>>>>> Yes I am adding new virtual hosts when ever I want to release a new >>>>>> version of that services lets say initially my virtualhost config will >>>>>> be >>>>>> like >>>>>> >>>>>> #Students Webservice Config >>>>>> Listen 9002 <VirtualHost *:9002> >>>>>> ServerName test.myapp.com >>>>>> ErrorLog /var/log/webservice_error.log >>>>>> WSGIPassAuthorization On >>>>>> WSGIDaemonProcess Tes9002 python-path=/home/uoadmin/releases/1.0.0 >>>>>> /students:/home/admin/releases/1.0.0/shared display-name=%{GROUP} >>>>>> WSGIProcessGroup Tes9002 WSGIApplicationGroup %{GLOBAL} >>>>>> WSGIScriptAlias / /home/admin/releases/1.0.0/students/conf/wsgi.py >>>>>> <Directory /home/admin/releases/1.0.0/students/conf> >>>>>> <Files wsgi.py> Require all granted </Files> >>>>>> </Directory> >>>>>> </VirtualHost> >>>>>> >>>>>> When i want to go for new releases the down part is appended to above >>>>>> part >>>>>> >>>>>> <VirtualHost *:9002> >>>>>> ServerName dev.myapp.com >>>>>> ErrorLog /var/log/webservice_error.log >>>>>> WSGIPassAuthorization On >>>>>> WSGIDaemonProcess Dev9002 python- path=/home/uoadmin/releases/1.1.0 >>>>>> /students:/home/admin/releases/1.1.0/shared display-name=%{GROUP} >>>>>> WSGIProcessGroup Dev9002 WSGIApplicationGroup %{GLOBAL} >>>>>> WSGIScriptAlias / /home/admin/releases/1.1.0/students/conf/wsgi.py >>>>>> <Directory /home/admin/releases/1.1.0/students/conf> >>>>>> <Files wsgi.py> Require all granted </Files> >>>>>> </Directory> >>>>>> </VirtualHost> >>>>>> >>>>>> >>>>>> Now I am going to have two virtualhosts with two daemons 1st is >>>>>> already recognized by apache server where as second one is not yet >>>>>> recognized by apache server >>>>>> >>>>>> Thankyou, >>>>>> RajKumar >>>>>> >>>>>> On Thursday 25 July 2024 at 11:40:31 UTC+5:30 Graham Dumpleton wrote: >>>>>> >>>>>>> What is the reason for doing the graceful restart? Is it because you >>>>>>> are adding/removing virtual hosts, or making some other Apache config >>>>>>> change. >>>>>>> >>>>>>> You do not need to do a complete Apache restart if just want to >>>>>>> force a daemon process to restart, you can instead send the processes a >>>>>>> signal directly. From memory it is SIGUSR1 that triggers a graceful >>>>>>> restart >>>>>>> of processes, but you will need to confirm that. >>>>>>> >>>>>>> On 25 Jul 2024, at 3:28 PM, RajKumar Ambadipelli <arkki...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>> When i have 260 microservices those all are light weight >>>>>>> applications using same python interpreter and with django rest api >>>>>>> framework, and currently each application hosted on apache server usign >>>>>>> mod_wsgi daemon mode and my main problem is while making changes to one >>>>>>> of >>>>>>> application virtualhost other ongoing daemons are distured as i need to >>>>>>> reload or restart. >>>>>>> All those 260 services very light weight each listen to http request >>>>>>> on unique ports. >>>>>>> >>>>>>> ThankYou >>>>>>> RajKumar >>>>>>> >>>>>>> On Tuesday 23 July 2024 at 16:37:42 UTC+5:30 Graham Dumpleton wrote: >>>>>>> >>>>>>>> One can optimise embedded mode for better performance, but I would >>>>>>>> put a big caveat on that and say is only probably a good idea to >>>>>>>> tackle if >>>>>>>> you have the one web service. >>>>>>>> >>>>>>>> Running 260 micro services in one Apache httpd instance with >>>>>>>> mod_wsgi sounds rather scary either way. >>>>>>>> >>>>>>>> If you use mod_wsgi daemon mode where each micro service is in its >>>>>>>> own daemon process group (with a single process and small number of >>>>>>>> threads), then you might get away with it if these aren't high volume >>>>>>>> sites. That said, it is still a lot of managed daemon mode processes >>>>>>>> and >>>>>>>> not sure how well Apache will handle that, especially on restarts. >>>>>>>> >>>>>>>> Running them all in embedded mode would be a bad idea if each needs >>>>>>>> a separate Python interpreter context because the Apache worker >>>>>>>> process >>>>>>>> would be huge in size. If Apache httpd was configured for prefork MPM >>>>>>>> it >>>>>>>> would be even worse because you would have a potentially large number >>>>>>>> of >>>>>>>> worker processes since all are single thread. You also run a big risk >>>>>>>> with >>>>>>>> micro services interfering with each other in strange ways if running >>>>>>>> in >>>>>>>> different sub interpreter contexts of the one process due to how >>>>>>>> Python >>>>>>>> imports C extensions, and process wide environment variables work. >>>>>>>> Various >>>>>>>> third party Python packages with C extensions will not even work in >>>>>>>> Python >>>>>>>> sub interpreters (eg., anything related to numpy). >>>>>>>> >>>>>>>> You definitely want event or worker MPM, but even then, for 260 >>>>>>>> micro services, if they need separate Python interpreter context I >>>>>>>> can't >>>>>>>> really recommend it still because of size concerns for processes and >>>>>>>> potential cross sub interpreter interference. >>>>>>>> >>>>>>>> So the question is whether when you said 260 micro services you >>>>>>>> really mean independent web applications, or whether you just mean you >>>>>>>> have >>>>>>>> 260 different unique HTTP handlers as part of the one application, and >>>>>>>> thus >>>>>>>> in the same Python interpreter context. >>>>>>>> >>>>>>>> When people talk about such large number of micro services, usually >>>>>>>> you would not be aiming to host them in a single Apache instance but >>>>>>>> would >>>>>>>> instead be looking at running something which can handle things at >>>>>>>> scale >>>>>>>> like Kubernetes and creating separate deployments for them in that, >>>>>>>> relying >>>>>>>> on the ingress routing Kubernetes provides to get traffic to the >>>>>>>> appropriate micro service. >>>>>>>> >>>>>>>> Graham >>>>>>>> >>>>>>>> On 23 Jul 2024, at 7:13 PM, RajKumar Ambadipelli < >>>>>>>> arkki...@gmail.com> wrote: >>>>>>>> >>>>>>>> mod_wsgi in embeded mode allows graceful restart, >>>>>>>> What are the potential issues that I will face if I use mod_wsgi in >>>>>>>> embedded mode instead of daemon mode, >>>>>>>> I have to host around 260 python micro services. >>>>>>>> >>>>>>>> I have saw your blog on 'why are you using mod_wsgi in embedded >>>>>>>> mode?' But, I unable to understand it very well in that you mentioned >>>>>>>> if we >>>>>>>> configure mpm settings correctly then mod_wsgi in embedded mode is >>>>>>>> better >>>>>>>> than daemon mode but not mentioned any configurations. >>>>>>>> >>>>>>>> Thanking you, >>>>>>>> RajKumar >>>>>>>> >>>>>>>> On Tuesday 23 July 2024 at 13:04:50 UTC+5:30 Graham Dumpleton wrote: >>>>>>>> >>>>>>>>> On 23 Jul 2024, at 4:09 PM, RajKumar Ambadipelli <arkki...@ >>>>>>>>> gmail.com> wrote: >>>>>>>>> >>>>>>>>> I am using Apache Server with mod_wsgi for hosting my python >>>>>>>>> django applications. Versions: Python 3.9.18 Server version: >>>>>>>>> Apache/2.4.57 >>>>>>>>> mod-wsgi==4.7.1 >>>>>>>>> >>>>>>>>> One of my application virtual host configuration with two >>>>>>>>> different versions: >>>>>>>>> >>>>>>>>> ... >>>>>>>>> >>>>>>>>> So, When the source code is modified I can referesh the wsgi >>>>>>>>> daemon using touch /home/uoadmin/releases/1.1.0/students/conf/wsgi.py >>>>>>>>> touch >>>>>>>>> /home/uoadmin/releases/1.0.0/students/conf/wsgi.py But when I added >>>>>>>>> new >>>>>>>>> virtualhost to the above configuration file or else when I modify >>>>>>>>> above >>>>>>>>> file the apache server unable to recognize modifications made the >>>>>>>>> existing >>>>>>>>> virtualhost or newly added virtualhost until doing apachectl graceful >>>>>>>>> (or) >>>>>>>>> apachectl restart (or) systemctl reload httpd but all the commands >>>>>>>>> above >>>>>>>>> killing the ongoing requests forcefully directly terminating them. >>>>>>>>> >>>>>>>>> How to handle above situation. >>>>>>>>> >>>>>>>>> I want to know how will apache server recognize modifications to >>>>>>>>> virtualhost or newly added virtual host without reloading or >>>>>>>>> restarting. >>>>>>>>> >>>>>>>>> It can't, Apache httpd requires you to perform a restart (reload) >>>>>>>>> in order to read changes to the Apache configuration files. That is >>>>>>>>> how it >>>>>>>>> works. >>>>>>>>> >>>>>>>>> If above is not possible then is there anyway for restarting or >>>>>>>>> reloading apache server gracefully that is without terminating or >>>>>>>>> killing >>>>>>>>> other ongoing requests or daemons while using apache server + >>>>>>>>> mod_wsgi for >>>>>>>>> serving python with django? >>>>>>>>> >>>>>>>>> Unfortunately not. The way Apache httpd manages the mod_wsgi >>>>>>>>> daemon processes it will force a restart of those as well and even >>>>>>>>> though >>>>>>>>> Apache has a concept of graceful restart for it's own worker child >>>>>>>>> processes, it doesn't extend that to managed process like the >>>>>>>>> mod_wsgi >>>>>>>>> daemon process and always restarts them immediately even when it is a >>>>>>>>> graceful restart. There is nothing that can be done about this. >>>>>>>>> >>>>>>>>> The only way you could handle it if you need to be able to freely >>>>>>>>> restart the main Apache server and have it not affect your Python web >>>>>>>>> applications, is to run the Python web applications in distinct >>>>>>>>> secondary >>>>>>>>> web server processes and use the main Apache server to only proxy >>>>>>>>> requests >>>>>>>>> through to the secondary web servers. >>>>>>>>> >>>>>>>>> For the second web servers you could use mod_wsgi-express to make >>>>>>>>> things easier, but you could also just not use mod_wsgi for the >>>>>>>>> secondary >>>>>>>>> web servers and use gunicorn or some other standalone Python >>>>>>>>> WSGI/asyncio >>>>>>>>> web server. >>>>>>>>> >>>>>>>>> Graham >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "modwsgi" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to modwsgi+u...@googlegroups.com. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/modwsgi/d28663bc-a143-4e4f-949d-38e065c5ac9fn%40googlegroups.com >>>>>>>> >>>>>>>> <https://groups.google.com/d/msgid/modwsgi/d28663bc-a143-4e4f-949d-38e065c5ac9fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "modwsgi" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to modwsgi+u...@googlegroups.com. >>>>>>> >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/modwsgi/1fffb2f7-ed8a-4d88-a52b-00e7e82e98d5n%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/modwsgi/1fffb2f7-ed8a-4d88-a52b-00e7e82e98d5n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>>> >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "modwsgi" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to modwsgi+u...@googlegroups.com. >>>>>> >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/modwsgi/27697b57-c903-4881-bddd-691060d62b47n%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/modwsgi/27697b57-c903-4881-bddd-691060d62b47n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>>> >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "modwsgi" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to modwsgi+u...@googlegroups.com. >>>>> >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/modwsgi/0794a799-7f70-4d68-affe-b2b7a4f43529n%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/modwsgi/0794a799-7f70-4d68-affe-b2b7a4f43529n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> >>>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "modwsgi" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to modwsgi+u...@googlegroups.com. >>> >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/modwsgi/7edb7907-915b-41a6-a581-097ea1b87dc0n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/modwsgi/7edb7907-915b-41a6-a581-097ea1b87dc0n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> >>> > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to modwsgi+u...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/modwsgi/618674de-ba2c-49df-8df9-10db6624df7fn%40googlegroups.com > > <https://groups.google.com/d/msgid/modwsgi/618674de-ba2c-49df-8df9-10db6624df7fn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/82829fd0-08c9-44b4-8f12-b42f0b7c6db9n%40googlegroups.com.
