Understood. Thankyou RajKumar
On Monday 5 August 2024 at 09:39:47 UTC+5:30 Graham Dumpleton wrote: > Memory will still be claimed by the process. > > I already told you to use: > > inactivity-timeout=30 > > on WSGIDaemonProcess to have the daemon processes restart after non > activity for a while so memory will return to base amount for Python > interpreter. > > Graham > > On 5 Aug 2024, at 2:06 PM, RajKumar Ambadipelli <arkki...@gmail.com> > wrote: > > After processing request with a wsgi daemon what happens to cpu and memory > footprints that is the cpu spike's that got during processing request will > be gone after processing request but what about the memory footprint > (ram usage) does it will remain in the cache even after completing > requests if so how to gracefully remove it without disturbing the ongoing > request is it even possible. > > RajKumar > On Thursday 1 August 2024 at 12:35:09 UTC+5:30 Graham Dumpleton wrote: > >> The number of process/threads defined in WSGIDaemonProcess is completely >> independent of MPM settings and which MPM module (prefork/event/worker) is >> being used. >> >> Yes having 15 threads means 15 requests can be handled concurrently, but >> do not be deceived in thinking that is the maximum throughput in requests >> per second you can handle and that you need to set it higher. In fact I >> actually recommend people drop it down to 5 threads per process, as 3-5 >> threads process is a bit of a sweet spot for getting best performance for >> one process. >> >> For more background I suggest you watch the following conference talks. >> >> [image: maxresdefault.jpg] >> >> Secrets of a WSGI master. <https://www.youtube.com/watch?v=H6Q3l11fjU0> >> youtube.com <https://www.youtube.com/watch?v=H6Q3l11fjU0> >> <https://www.youtube.com/watch?v=H6Q3l11fjU0> >> >> [image: maxresdefault.jpg] >> >> Using benchmarks to understand how WSGI servers work. by Graham Dumpleton >> <https://www.youtube.com/watch?v=SGleKfigMsk> >> youtube.com <https://www.youtube.com/watch?v=SGleKfigMsk> >> <https://www.youtube.com/watch?v=SGleKfigMsk> >> >> [image: hqdefault.jpg] >> >> Making Apache suck less for hosting Python web applications. >> <https://www.youtube.com/watch?v=k6Erh7oHvns> >> youtube.com <https://www.youtube.com/watch?v=k6Erh7oHvns> >> <https://www.youtube.com/watch?v=k6Erh7oHvns> >> >> As explained in the videos, even if all request threads in a daemon >> process are occupied, then requests will queue up within the context of the >> Apache worker process which proxy to the mod_wsgi daemon processes. The MPM >> settings control those worker processes, and is typical for those to have >> higher capacity for concurrent requests, thus allowing queueing up of >> requests. There can also be a backlog of requests connecting to Apache as >> well. These topics are described in the videos. >> >> Graham >> >> On 1 Aug 2024, at 4:56 PM, RajKumar Ambadipelli <arkki...@gmail.com> >> wrote: >> >> If we don't mention options like no.of process and no.of threads in >> WSGIDaemonProcess directive by default it will consider 1 process with 15 >> threads in mpm settings as event prefork right. >> Does the above configuration helps in serving 15 request concurrently. If >> so, then what happens when there are more no.of concurrent requests hit our >> web application. >> Does 15 threads means it serves 15 requests concurrently. >> >> RajKumar >> >> On Wednesday 31 July 2024 at 12:47:09 UTC+5:30 RajKumar Ambadipelli wrote: >> >>> Thankyou for clarifying. >>> >>> RajKumar >>> >>> On Wednesday 31 July 2024 at 11:56:48 UTC+5:30 Graham Dumpleton wrote: >>> >>>> If you define a daemon process group, the number of processes defined >>>> for the group will always be started and running. >>>> >>>> With the way you have configured things your Python web application >>>> code is only loaded lazily by a process in a daemon process group when the >>>> first request arrives which is to be handled by that process. >>>> >>>> Thus, prior to any request being received, the memory footprint of the >>>> processes should be similar to that of running an empty Python interpreter. >>>> >>>> If your web applications are infrequently accessed and you want to >>>> minimise memory usage, then add to WSGIDaemonProcess the option: >>>> >>>> inactivity-timeout=30 >>>> >>>> See >>>> https://modwsgi.readthedocs.io/en/master/configuration-directives/WSGIDaemonProcess.html >>>> for >>>> details, but basically what it means is that the process will be restarted >>>> when idle and will revert memory usage by the process back to the base >>>> level. >>>> >>>> Do note however that if you Python web application takes a long time to >>>> load, then this may be noticeable to users if the Python web application >>>> code is going to have to be reloaded frequently. >>>> >>>> As to CPU usage, the process if not handling requests will be waiting >>>> on the socket on which requests are listening, so it should not be using >>>> any CPU at that time. >>>> >>>> There are other options to WSGIDaemonProcess you could play with to >>>> recycle the process periodically, but what might be appropriate really >>>> depends on your specific web application so I can't give concrete >>>> suggestions of what to use. >>>> >>>> Graham >>>> >>>> On 31 Jul 2024, at 2:55 PM, RajKumar Ambadipelli <arkki...@gmail.com> >>>> wrote: >>>> >>>> What exactly I want is I want to know WSGIDaemonProcess which are ideal >>>> not receiving any requests and not serving response but only declared, How >>>> does this WSGIDaemonProcess effect my resources like memory and cpu. >>>> >>>> On Wednesday 31 July 2024 at 10:22:23 UTC+5:30 RajKumar Ambadipelli >>>> wrote: >>>> >>>>> Creating multiple virtualhost with different mod_wsgi daemons I meant >>>>> 'WSGIDaemonProcess' >>>>> and not using it that is it will not server and requests but only >>>>> declared does this effect my resources like memory and CPU. >>>>> On Monday 29 July 2024 at 12:45:21 UTC+5:30 Graham Dumpleton wrote: >>>>> >>>>>> Comes down to how much memory and CPU your machine has, plus how much >>>>>> traffic the sites get. So will depend and you will just have to see how >>>>>> it >>>>>> goes. >>>>>> >>>>>> On 29 Jul 2024, at 5:12 PM, RajKumar Ambadipelli <arkki...@gmail.com> >>>>>> wrote: >>>>>> >>>>>> Yeah it is working fine what I did was >>>>>> >>>>>> step1. Unlinked existing symlink >>>>>> Step2. Create new symlink >>>>>> Step3. Managed permissions and shell context of files >>>>>> >>>>>> Therefore no >>>>>> reload or restart is needed. >>>>>> >>>>>> But I have doubt up to how many mod_wsgi daemons we can efficiently >>>>>> without any disturbance and up to what extent we can trust this >>>>>> deployment >>>>>> process using mod_wsgi in daemon mode for production environment. >>>>>> >>>>>> Thankyou, >>>>>> RajKumar >>>>>> On Thursday 25 July 2024 at 13:04:18 UTC+5:30 RajKumar Ambadipelli >>>>>> wrote: >>>>>> >>>>>>> Sorry that uoadmin is actually admin. >>>>>>> >>>>>>> Ok I will try this one. >>>>>>> >>>>>>> Thankyou, >>>>>>> RajKumar >>>>>>> >>>>>>> On Thursday 25 July 2024 at 12:59:13 UTC+5:30 Graham Dumpleton wrote: >>>>>>> >>>>>>>> You have two different directories which may complicate it a >>>>>>>> little, but what you can do is have something like: >>>>>>>> >>>>>>>> Listen 9002 <VirtualHost *:9002> >>>>>>>> ServerName test.myapp.com >>>>>>>> ErrorLog /var/log/webservice_error.log >>>>>>>> WSGIPassAuthorization On >>>>>>>> WSGIDaemonProcess Tes9002 >>>>>>>> python-path=/home/uoadmin/releases/test/students:/home/admin/releases/test/shared >>>>>>>> >>>>>>>> display-name=%{GROUP} >>>>>>>> WSGIProcessGroup Tes9002 WSGIApplicationGroup %{GLOBAL} >>>>>>>> WSGIScriptAlias / /home/admin/releases/test/students/conf/wsgi.py >>>>>>>> <Directory /home/admin/releases/test/students/conf> >>>>>>>> <Files wsgi.py> Require all granted </Files> >>>>>>>> </Directory> >>>>>>>> </VirtualHost> >>>>>>>> >>>>>>>> <VirtualHost *:9002> >>>>>>>> ServerName dev.myapp.com >>>>>>>> ErrorLog /var/log/webservice_error.log >>>>>>>> WSGIPassAuthorization On >>>>>>>> WSGIDaemonProcess Dev9002 >>>>>>>> python-path=/home/uoadmin/releases/dev/students:/home/admin/releases/dev/shared >>>>>>>> >>>>>>>> display-name=%{GROUP} >>>>>>>> WSGIProcessGroup Dev9002 WSGIApplicationGroup %{GLOBAL} >>>>>>>> WSGIScriptAlias / /home/admin/releases/dev/students/conf/wsgi.py >>>>>>>> <Directory /home/admin/releases/dev/students/conf> >>>>>>>> <Files wsgi.py> Require all granted </Files> >>>>>>>> </Directory> >>>>>>>> </VirtualHost> >>>>>>>> >>>>>>>> In this case /home/admin/releases/dev would actually be a symlink >>>>>>>> to the versioned directory. >>>>>>>> >>>>>>>> Not sure why have separate uoadmin directory, but also have >>>>>>>> /home/uoadmin/releases/dev as symlink to the versioned directory. >>>>>>>> >>>>>>>> To change what is used, remove/recreate symlinks so point to the >>>>>>>> directory for the new version. >>>>>>>> >>>>>>>> That the wsgi.py has changed should trigger a process restart if >>>>>>>> auto reloading is on (default). >>>>>>>> >>>>>>>> Alternatively could disable auto reloading and manually send >>>>>>>> SIGUSR1 to process to trigger a graceful reload after changing the >>>>>>>> symlink. >>>>>>>> >>>>>>>> That all said, I can't remember if you might have to configure >>>>>>>> Apache to tell it to allow following symlinks (Options >>>>>>>> FollowSymLinks). If it were static file handling would definitely be >>>>>>>> needed, but can't remember if would be required in this case where is >>>>>>>> WSGI >>>>>>>> application handling things. >>>>>>>> >>>>>>>> Graham >>>>>>>> >>>>>>>> On 25 Jul 2024, at 5:21 PM, RajKumar Ambadipelli <arkki...@ >>>>>>>> gmail.com> wrote: >>>>>>>> >>>>>>>> Ok, If I have only two virtualhosts all the time and I am going to >>>>>>>> change only python-path will that work with the direct signal using >>>>>>>> SIGUSR1. >>>>>>>> >>>>>>>> If the above is possible then I think I have some kind of solution. >>>>>>>> >>>>>>>> Thankyou, >>>>>>>> RajKumar >>>>>>>> >>>>>>>> On Thursday 25 July 2024 at 12:35:09 UTC+5:30 Graham Dumpleton >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Are you always using the same two virtual host server names and >>>>>>>>> just updating the version number in the paths? >>>>>>>>> >>>>>>>>> On 25 Jul 2024, at 4:21 PM, RajKumar Ambadipelli <arkki...@ >>>>>>>>> gmail.com> wrote: >>>>>>>>> >>>>>>>>> Yes I am adding new virtual hosts when ever I want to release a >>>>>>>>> new version of that services lets say initially my virtualhost config >>>>>>>>> will >>>>>>>>> be like >>>>>>>>> >>>>>>>>> #Students Webservice Config >>>>>>>>> Listen 9002 <VirtualHost *:9002> >>>>>>>>> ServerName test.myapp.com >>>>>>>>> ErrorLog /var/log/webservice_error.log >>>>>>>>> WSGIPassAuthorization On >>>>>>>>> WSGIDaemonProcess Tes9002 python-path=/home/uoadmin/releases/1.0 >>>>>>>>> .0/students:/home/admin/releases/1.0.0/shared >>>>>>>>> display-name=%{GROUP} >>>>>>>>> WSGIProcessGroup Tes9002 WSGIApplicationGroup %{GLOBAL} >>>>>>>>> WSGIScriptAlias / /home/admin/releases/1.0.0 >>>>>>>>> /students/conf/wsgi.py >>>>>>>>> <Directory /home/admin/releases/1.0.0/students/conf> >>>>>>>>> <Files wsgi.py> Require all granted </Files> >>>>>>>>> </Directory> >>>>>>>>> </VirtualHost> >>>>>>>>> >>>>>>>>> When i want to go for new releases the down part is appended to >>>>>>>>> above part >>>>>>>>> >>>>>>>>> <VirtualHost *:9002> >>>>>>>>> ServerName dev.myapp.com >>>>>>>>> ErrorLog /var/log/webservice_error.log >>>>>>>>> WSGIPassAuthorization On >>>>>>>>> WSGIDaemonProcess Dev9002 python- path=/home/uoadmin/releases/1.1 >>>>>>>>> .0/students:/home/admin/releases/1.1.0/shared >>>>>>>>> display-name=%{GROUP} >>>>>>>>> WSGIProcessGroup Dev9002 WSGIApplicationGroup %{GLOBAL} >>>>>>>>> WSGIScriptAlias / /home/admin/releases/1.1.0 >>>>>>>>> /students/conf/wsgi.py >>>>>>>>> <Directory /home/admin/releases/1.1.0/students/conf> >>>>>>>>> <Files wsgi.py> Require all granted </Files> >>>>>>>>> </Directory> >>>>>>>>> </VirtualHost> >>>>>>>>> >>>>>>>>> >>>>>>>>> Now I am going to have two virtualhosts with two daemons 1st is >>>>>>>>> already recognized by apache server where as second one is not yet >>>>>>>>> recognized by apache server >>>>>>>>> >>>>>>>>> Thankyou, >>>>>>>>> RajKumar >>>>>>>>> >>>>>>>>> On Thursday 25 July 2024 at 11:40:31 UTC+5:30 Graham Dumpleton >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> What is the reason for doing the graceful restart? Is it because >>>>>>>>>> you are adding/removing virtual hosts, or making some other Apache >>>>>>>>>> config >>>>>>>>>> change. >>>>>>>>>> >>>>>>>>>> You do not need to do a complete Apache restart if just want to >>>>>>>>>> force a daemon process to restart, you can instead send the >>>>>>>>>> processes a >>>>>>>>>> signal directly. From memory it is SIGUSR1 that triggers a graceful >>>>>>>>>> restart >>>>>>>>>> of processes, but you will need to confirm that. >>>>>>>>>> >>>>>>>>>> On 25 Jul 2024, at 3:28 PM, RajKumar Ambadipelli <arkki...@ >>>>>>>>>> gmail.com> wrote: >>>>>>>>>> >>>>>>>>>> When i have 260 microservices those all are light weight >>>>>>>>>> applications using same python interpreter and with django rest api >>>>>>>>>> framework, and currently each application hosted on apache server >>>>>>>>>> usign >>>>>>>>>> mod_wsgi daemon mode and my main problem is while making changes to >>>>>>>>>> one of >>>>>>>>>> application virtualhost other ongoing daemons are distured as i need >>>>>>>>>> to >>>>>>>>>> reload or restart. >>>>>>>>>> All those 260 services very light weight each listen to http >>>>>>>>>> request on unique ports. >>>>>>>>>> >>>>>>>>>> ThankYou >>>>>>>>>> RajKumar >>>>>>>>>> >>>>>>>>>> On Tuesday 23 July 2024 at 16:37:42 UTC+5:30 Graham Dumpleton >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> One can optimise embedded mode for better performance, but I >>>>>>>>>>> would put a big caveat on that and say is only probably a good idea >>>>>>>>>>> to >>>>>>>>>>> tackle if you have the one web service. >>>>>>>>>>> >>>>>>>>>>> Running 260 micro services in one Apache httpd instance with >>>>>>>>>>> mod_wsgi sounds rather scary either way. >>>>>>>>>>> >>>>>>>>>>> If you use mod_wsgi daemon mode where each micro service is in >>>>>>>>>>> its own daemon process group (with a single process and small >>>>>>>>>>> number of >>>>>>>>>>> threads), then you might get away with it if these aren't high >>>>>>>>>>> volume >>>>>>>>>>> sites. That said, it is still a lot of managed daemon mode >>>>>>>>>>> processes and >>>>>>>>>>> not sure how well Apache will handle that, especially on restarts. >>>>>>>>>>> >>>>>>>>>>> Running them all in embedded mode would be a bad idea if each >>>>>>>>>>> needs a separate Python interpreter context because the Apache >>>>>>>>>>> worker >>>>>>>>>>> process would be huge in size. If Apache httpd was configured for >>>>>>>>>>> prefork >>>>>>>>>>> MPM it would be even worse because you would have a potentially >>>>>>>>>>> large >>>>>>>>>>> number of worker processes since all are single thread. You also >>>>>>>>>>> run a big >>>>>>>>>>> risk with micro services interfering with each other in strange >>>>>>>>>>> ways if >>>>>>>>>>> running in different sub interpreter contexts of the one process >>>>>>>>>>> due to how >>>>>>>>>>> Python imports C extensions, and process wide environment variables >>>>>>>>>>> work. >>>>>>>>>>> Various third party Python packages with C extensions will not even >>>>>>>>>>> work in >>>>>>>>>>> Python sub interpreters (eg., anything related to numpy). >>>>>>>>>>> >>>>>>>>>>> You definitely want event or worker MPM, but even then, for 260 >>>>>>>>>>> micro services, if they need separate Python interpreter context I >>>>>>>>>>> can't >>>>>>>>>>> really recommend it still because of size concerns for processes >>>>>>>>>>> and >>>>>>>>>>> potential cross sub interpreter interference. >>>>>>>>>>> >>>>>>>>>>> So the question is whether when you said 260 micro services you >>>>>>>>>>> really mean independent web applications, or whether you just mean >>>>>>>>>>> you have >>>>>>>>>>> 260 different unique HTTP handlers as part of the one application, >>>>>>>>>>> and thus >>>>>>>>>>> in the same Python interpreter context. >>>>>>>>>>> >>>>>>>>>>> When people talk about such large number of micro services, >>>>>>>>>>> usually you would not be aiming to host them in a single Apache >>>>>>>>>>> instance >>>>>>>>>>> but would instead be looking at running something which can handle >>>>>>>>>>> things >>>>>>>>>>> at scale like Kubernetes and creating separate deployments for them >>>>>>>>>>> in >>>>>>>>>>> that, relying on the ingress routing Kubernetes provides to get >>>>>>>>>>> traffic to >>>>>>>>>>> the appropriate micro service. >>>>>>>>>>> >>>>>>>>>>> Graham >>>>>>>>>>> >>>>>>>>>>> On 23 Jul 2024, at 7:13 PM, RajKumar Ambadipelli <arkki...@ >>>>>>>>>>> gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>> mod_wsgi in embeded mode allows graceful restart, >>>>>>>>>>> What are the potential issues that I will face if I use mod_wsgi >>>>>>>>>>> in embedded mode instead of daemon mode, >>>>>>>>>>> I have to host around 260 python micro services. >>>>>>>>>>> >>>>>>>>>>> I have saw your blog on 'why are you using mod_wsgi in embedded >>>>>>>>>>> mode?' But, I unable to understand it very well in that you >>>>>>>>>>> mentioned if we >>>>>>>>>>> configure mpm settings correctly then mod_wsgi in embedded mode is >>>>>>>>>>> better >>>>>>>>>>> than daemon mode but not mentioned any configurations. >>>>>>>>>>> >>>>>>>>>>> Thanking you, >>>>>>>>>>> RajKumar >>>>>>>>>>> >>>>>>>>>>> On Tuesday 23 July 2024 at 13:04:50 UTC+5:30 Graham Dumpleton >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> On 23 Jul 2024, at 4:09 PM, RajKumar Ambadipelli <arkki...@ >>>>>>>>>>>> gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>> I am using Apache Server with mod_wsgi for hosting my python >>>>>>>>>>>> django applications. Versions: Python 3.9.18 Server version: >>>>>>>>>>>> Apache/2.4.57 >>>>>>>>>>>> mod-wsgi==4.7.1 >>>>>>>>>>>> >>>>>>>>>>>> One of my application virtual host configuration with two >>>>>>>>>>>> different versions: >>>>>>>>>>>> >>>>>>>>>>>> ... >>>>>>>>>>>> >>>>>>>>>>>> So, When the source code is modified I can referesh the wsgi >>>>>>>>>>>> daemon using touch >>>>>>>>>>>> /home/uoadmin/releases/1.1.0/students/conf/wsgi.py touch >>>>>>>>>>>> /home/uoadmin/releases/1.0.0/students/conf/wsgi.py But when I >>>>>>>>>>>> added new >>>>>>>>>>>> virtualhost to the above configuration file or else when I modify >>>>>>>>>>>> above >>>>>>>>>>>> file the apache server unable to recognize modifications made the >>>>>>>>>>>> existing >>>>>>>>>>>> virtualhost or newly added virtualhost until doing apachectl >>>>>>>>>>>> graceful (or) >>>>>>>>>>>> apachectl restart (or) systemctl reload httpd but all the commands >>>>>>>>>>>> above >>>>>>>>>>>> killing the ongoing requests forcefully directly terminating them. >>>>>>>>>>>> >>>>>>>>>>>> How to handle above situation. >>>>>>>>>>>> >>>>>>>>>>>> I want to know how will apache server recognize modifications >>>>>>>>>>>> to virtualhost or newly added virtual host without reloading or >>>>>>>>>>>> restarting. >>>>>>>>>>>> >>>>>>>>>>>> It can't, Apache httpd requires you to perform a restart >>>>>>>>>>>> (reload) in order to read changes to the Apache configuration >>>>>>>>>>>> files. That >>>>>>>>>>>> is how it works. >>>>>>>>>>>> >>>>>>>>>>>> If above is not possible then is there anyway for restarting or >>>>>>>>>>>> reloading apache server gracefully that is without terminating or >>>>>>>>>>>> killing >>>>>>>>>>>> other ongoing requests or daemons while using apache server + >>>>>>>>>>>> mod_wsgi for >>>>>>>>>>>> serving python with django? >>>>>>>>>>>> >>>>>>>>>>>> Unfortunately not. The way Apache httpd manages the mod_wsgi >>>>>>>>>>>> daemon processes it will force a restart of those as well and even >>>>>>>>>>>> though >>>>>>>>>>>> Apache has a concept of graceful restart for it's own worker child >>>>>>>>>>>> processes, it doesn't extend that to managed process like the >>>>>>>>>>>> mod_wsgi >>>>>>>>>>>> daemon process and always restarts them immediately even when it >>>>>>>>>>>> is a >>>>>>>>>>>> graceful restart. There is nothing that can be done about this. >>>>>>>>>>>> >>>>>>>>>>>> The only way you could handle it if you need to be able to >>>>>>>>>>>> freely restart the main Apache server and have it not affect your >>>>>>>>>>>> Python >>>>>>>>>>>> web applications, is to run the Python web applications in >>>>>>>>>>>> distinct >>>>>>>>>>>> secondary web server processes and use the main Apache server to >>>>>>>>>>>> only proxy >>>>>>>>>>>> requests through to the secondary web servers. >>>>>>>>>>>> >>>>>>>>>>>> For the second web servers you could use mod_wsgi-express to >>>>>>>>>>>> make things easier, but you could also just not use mod_wsgi for >>>>>>>>>>>> the >>>>>>>>>>>> secondary web servers and use gunicorn or some other standalone >>>>>>>>>>>> Python >>>>>>>>>>>> WSGI/asyncio web server. >>>>>>>>>>>> >>>>>>>>>>>> Graham >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "modwsgi" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to modwsgi+u...@googlegroups.com. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/modwsgi/d28663bc-a143-4e4f-949d-38e065c5ac9fn%40googlegroups.com >>>>>>>>>>> >>>>>>>>>>> <https://groups.google.com/d/msgid/modwsgi/d28663bc-a143-4e4f-949d-38e065c5ac9fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "modwsgi" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to modwsgi+u...@googlegroups.com. >>>>>>>>>> >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/modwsgi/1fffb2f7-ed8a-4d88-a52b-00e7e82e98d5n%40googlegroups.com >>>>>>>>>> >>>>>>>>>> <https://groups.google.com/d/msgid/modwsgi/1fffb2f7-ed8a-4d88-a52b-00e7e82e98d5n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "modwsgi" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to modwsgi+u...@googlegroups.com. >>>>>>>>> >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/modwsgi/27697b57-c903-4881-bddd-691060d62b47n%40googlegroups.com >>>>>>>>> >>>>>>>>> <https://groups.google.com/d/msgid/modwsgi/27697b57-c903-4881-bddd-691060d62b47n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "modwsgi" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to modwsgi+u...@googlegroups.com. >>>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/modwsgi/0794a799-7f70-4d68-affe-b2b7a4f43529n%40googlegroups.com >>>>>>>> >>>>>>>> <https://groups.google.com/d/msgid/modwsgi/0794a799-7f70-4d68-affe-b2b7a4f43529n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "modwsgi" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to modwsgi+u...@googlegroups.com. >>>>>> >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/modwsgi/7edb7907-915b-41a6-a581-097ea1b87dc0n%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/modwsgi/7edb7907-915b-41a6-a581-097ea1b87dc0n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>>> >>>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "modwsgi" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to modwsgi+u...@googlegroups.com. >>>> >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/modwsgi/618674de-ba2c-49df-8df9-10db6624df7fn%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/modwsgi/618674de-ba2c-49df-8df9-10db6624df7fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> >>>> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to modwsgi+u...@googlegroups.com. >> >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/modwsgi/f08337f6-63a6-4fc4-a630-f437187b2b2en%40googlegroups.com >> >> <https://groups.google.com/d/msgid/modwsgi/f08337f6-63a6-4fc4-a630-f437187b2b2en%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> >> > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to modwsgi+u...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/modwsgi/7433c99a-c564-4a3c-8474-349200543159n%40googlegroups.com > > <https://groups.google.com/d/msgid/modwsgi/7433c99a-c564-4a3c-8474-349200543159n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/289bc037-3387-45ee-a81d-2045f22378fcn%40googlegroups.com.
