Yes, I could verify this. Threads are spawned with VSZ=185816 kB. The swap is not touched. The question now is, if that is a neglectable flaw or something that becomes nasty some day on other aspects. Anyway, it still deserves more investigation because VSZ problems are no common problems of standard software under Linux. For example, the apache gurus (apache 1.3.27 source) refrained from pthreads and made use of pthread_mutex only. This is stuff for more investigation.
-----Urspr�ngliche Nachricht----- Von: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Auftrag von Fr�d�ric Giudicelli Gesendet: Freitag, 30. Januar 2004 09:04 An: [EMAIL PROTECTED] Betreff: Re: Many instances of newpki-server running Well, after browsing google for a while, it seems like this is normal, as apparently pthreads uses a lot of virtual memory but never commits it. Here is why, each thread is created with a stack of 8388608B, so 90 * 8388608B = 720mB. There, you have your explaintation :) Fr�d�ric Giudicelli wrote: > Oops, never mind, the code I sent is wrong, it should be: > for(i=0; i<90; i++) > instead of > for(i=0; i<50; i++) > > With the 90 threads, which is pretty much the number I have with my 8 > entities, I get a VSZ of 739060kB, so mysql is not involved in this > mess, it's all pthread's fault! :) > > > Fr�d�ric Giudicelli wrote: > >> Ok, so after a lot of investigastions, I came to this conclusion: >> either pthread sucks or VSZ is no relevent.... >> >> Here is a little testing program (test.cpp): >> >> #include <unistd.h> >> #include <pthread.h> >> #include <stdio.h> >> >> void * tmp_ThreadProc(void * Param) >> { >> for(;;) >> { >> usleep(1000000); >> printf("%d\n", pthread_self()); >> } >> } >> >> int main(int argc, char ** argv) >> { >> pthread_attr_t attr; >> pthread_t m_thread_id[90]; >> int i; >> >> pthread_attr_init(&attr); >> pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE); >> >> for(i=0; i<50; i++) >> { >> if(pthread_create(&m_thread_id[i], &attr, &tmp_ThreadProc, NULL)) >> { >> return 1; >> } >> } >> pthread_attr_destroy(&attr); >> >> pthread_join(m_thread_id[0], NULL); >> >> return 0; >> } >> >> >> To compile: >> gcc -o test -lpthread test.cpp >> >> After running it, on my system, it showed a VSZ of over 400mB.... >> There is nothing I can do if pthread has a weird implementation... >> >> The extra mB from the usual 700, are comming from libmysql, it seems >> to be using pthread as well, I'm testing it right now to see if there >> are any memory leaks comming from the way I use mysql. >> >> [EMAIL PROTECTED] wrote: >> >>> Hi Frederic, >>> >>> VSZ versus /proc/meminfo : >>> >>> /proc/meminf does not show the virt mem consumption because the mem was >>> malloced - but not used. The kernel is intelligent enough to just make a >>> reservation without assigning it when the malloced mem is not >>> referenced. >>> >>> So newpki_server seems to malloc or sbrk without using the mem. >>> >>> For example, the prog below mallocs VSZ=200MB of virtual mem. Without >>> the >>> memset below you cannot see effect in /proc/meminfo when the prog is >>> running. >>> >>>> From my standpoint newpki_server misbehaves in that regard. >>> >>> >>> >>> Wolf >>> ---------------- >>> >>> #include <unistd.h> >>> #include <stdio.h> >>> >>> main () { >>> >>> char c; >>> void *startof; >>> >>> startof = sbrk (1024*1024*200); >>> memset (startof,'a',1024*1024*200); /* just this bangs into the swap */ >>> read (0,&c,1); >>> } >>> >>> >>> -----Urspr�ngliche Nachricht----- >>> Von: [EMAIL PROTECTED] >>> [mailto:[EMAIL PROTECTED] Auftrag von Fr�d�ric Giudicelli >>> Gesendet: Donnerstag, 29. Januar 2004 01:51 >>> An: [EMAIL PROTECTED] >>> Betreff: Re: Many instances of newpki-server running >>> >>> >>> Ok, after a long investigation, here are the conclusions: there is no >>> memory leak and VSZ really is virtual :). >>> >>> Let me explain, for every thread, pthread "adds" 32mB to the vm. >>> I did this test, I reduced to 1 the number of connection threads >>> (instead of the 50 orignal ones) and bingo the VSZ reduced to a value >>> around 100mB, instead of the initial 700mB. >>> I then re-enabled the 50 connection threads, recompiled newpki_server, >>> and ran 'cat /proc/meminfo' before running newpki_server, here is the >>> output: >>> total: used: free: >>> Mem: 517894144 482705408 35188736 >>> Swap: 534601728 26198016 508403712 >>> >>> I then ran newpki_server (with 8 entities in it), waited a bit and >>> re-ran 'cat /proc/meminfo', here is the output >>> total: used: free: >>> Mem: 517894144 486785024 31109120 >>> Swap: 534601728 26198016 508403712 >>> >>> As you can see the swap didn't move, however 'cat /proc/`pidof >>> newpki_server`/status' tells me: >>> VmSize: 766168 kB >>> >>> As you can see, the info of VSZ is not to be taken for granted :) >>> >>> If you really want to do the test, install and run newpki_server on a >>> Win2000 server, configure it to have as many entities as in you linux >>> box, once the server is up and running run the "task manager", and look >>> at the memory usage of newpki_server.exe (in Windows the memory usage is >>> for the ram and the swap, there is no "bullshit" about its meaning), you >>> will see, it should be around 10mB. >>> >>> Regards, >>> -- >>> Fr�d�ric Giudicelli >>> http://www.newpki.org >>> >>> _____________________________________________________________________ >>> NewPKI http://www.newpki.org >>> User Support Mailing List [EMAIL PROTECTED] >>> Automated List Manager [EMAIL PROTECTED] >>> >>> _____________________________________________________________________ >>> NewPKI http://www.newpki.org >>> User Support Mailing List [EMAIL PROTECTED] >>> Automated List Manager [EMAIL PROTECTED] >> >> >> > -- Fr�d�ric Giudicelli http://www.newpki.org _____________________________________________________________________ NewPKI http://www.newpki.org User Support Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED] _____________________________________________________________________ NewPKI http://www.newpki.org User Support Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
