[naviserver-devel] new memory allocator to test

2006-12-19 Thread Vlad Seryakov
I put together another thing to test, it runs fine with my Naviserver and worked well with my old Tcl/Adp tests. Speed is less than zippy but better than Tcl or ptmalloc. But it does return memory back on thread exit, this made my naviserver nsd process always run around 170M instead of 800M wi

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Vlad Seryakov
On linux Tcl version of the test just crashes constantly in free, i have no other OSes here (gdb) bt #0 0xb7f0d410 in ?? () #1 0xb6cd1b78 in ?? () #2 0x0006 in ?? () #3 0x3746 in ?? () #4 0xb7d3f731 in raise () from /lib/libc.so.6 #5 0xb7d40f08 in abort () from /lib/libc.so.6 #6 0

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Zoran Vasiljevic
On 19.12.2006, at 20:42, Stephen Deasey wrote: On 12/19/06, Vlad Seryakov <[EMAIL PROTECTED]> wrote: Right, with Ns_ functions it does not crash. Zoran will be happy... :-) Not at all! So, I would like to know exactly how to reproduce the problem (what OS, machine, etc). Furthermore

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Stephen Deasey
On 12/19/06, Vlad Seryakov <[EMAIL PROTECTED]> wrote: Right, with Ns_ functions it does not crash. Zoran will be happy... :-)

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Vlad Seryakov
Right, with Ns_ functions it does not crash. Stephen Deasey wrote: On 12/19/06, Zoran Vasiljevic <[EMAIL PROTECTED]> wrote: On 19.12.2006, at 17:08, Vlad Seryakov wrote: I converted all to use pthreads directly instead of Tcl wrappers, and now it does not crash anymore. Will continue testing

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Stephen Deasey
On 12/19/06, Zoran Vasiljevic <[EMAIL PROTECTED]> wrote: On 19.12.2006, at 17:08, Vlad Seryakov wrote: > I converted all to use pthreads directly instead of Tcl wrappers, and > now it does not crash anymore. Will continue testing but it looks like > Tcl is the problem here, not ptmalloc Where

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Vlad Seryakov
I have no idea, i spent too much time on this still without realizing what i am doing and what to expect :-))) Zoran Vasiljevic wrote: On 19.12.2006, at 17:08, Vlad Seryakov wrote: I converted all to use pthreads directly instead of Tcl wrappers, and now it does not crash anymore. Will contin

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Zoran Vasiljevic
On 19.12.2006, at 17:08, Vlad Seryakov wrote: I converted all to use pthreads directly instead of Tcl wrappers, and now it does not crash anymore. Will continue testing but it looks like Tcl is the problem here, not ptmalloc Where does it crash? I see you are just using Tcl_CreateThread Tcl_M

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Vlad Seryakov
I converted all to use pthreads directly instead of Tcl wrappers, and now it does not crash anymore. Will continue testing but it looks like Tcl is the problem here, not ptmalloc Stephen Deasey wrote: On 12/19/06, Vlad Seryakov <[EMAIL PROTECTED]> wrote: yes, it crashes when number of threads

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Stephen Deasey
On 12/19/06, Vlad Seryakov <[EMAIL PROTECTED]> wrote: yes, it crashes when number of threads are more than 1 with any size but not all the time, sometimes i need to run it several times, looks like it is random, some combination, not sure of what. I guess we never got that high concurrency in Na

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Zoran Vasiljevic
On 19.12.2006, at 16:35, Vlad Seryakov wrote: yes, it crashes when number of threads are more than 1 with any size but not all the time, sometimes i need to run it several times, looks like it is random, some combination, not sure of what. I guess we never got that high concurrency in Navise

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Vlad Seryakov
yes, it crashes when number of threads are more than 1 with any size but not all the time, sometimes i need to run it several times, looks like it is random, some combination, not sure of what. I guess we never got that high concurrency in Naviserver, i wonder if AOL has randomm crashes. Ste

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Stephen Deasey
Is this really the shortest test case you can make for this problem? - Does it crash if you allocate blocks of size 1024 rather than random size? Does for me. Strip it out. - Does it crash if you run 2 threads instead of 4? Does for me. Strip it out. Some times it crashes, some times it doesn

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Vlad Seryakov
I was suspecting Linux malloc, looks like it has problems with high concurrency, i tried to replace MemAlloc/Fre with mmap/munmap, and it crashes as well. #define MemAlloc mmalloc #define MemFree(ptr) mfree(ptr, gSize) void *mmalloc(size_t size) { return mmap(NULL,size,PROT_READ|PROT_WRITE|PRO

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Zoran Vasiljevic
On 19.12.2006, at 16:15, Vlad Seryakov wrote: gdb may slow down concurrency, does it run without gdb, also does it run with solaris malloc? No problems. Runs with malloc and nedmalloc with or w/o gdb. The same on Mac.

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Vlad Seryakov
gdb may slow down concurrency, does it run without gdb, also does it run with solaris malloc? Zoran Vasiljevic wrote: On 19.12.2006, at 16:06, Vlad Seryakov wrote: Yes, please ( I appended the code to the nedmalloc test program and renamed their main to main1) bash-2.03$ gcc -O3 -o tclt

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Zoran Vasiljevic
On 19.12.2006, at 16:06, Vlad Seryakov wrote: Yes, please ( I appended the code to the nedmalloc test program and renamed their main to main1) bash-2.03$ gcc -O3 -o tcltest tcltest.c -lpthread -DNDEBUG - DTCL_THREADS -I/usr/local/include -L/usr/local/lib -ltcl8.4g bash-2.03$ gdb ./tcltes

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Vlad Seryakov
Yes, please Zoran Vasiljevic wrote: On 19.12.2006, at 15:57, Vlad Seryakov wrote: Zoran, can you test it on Solaris and OSX so we'd know that is not Linux related problem. I have a Tcl library compiled with nedmalloc and when I link against it and make #define MemAlloc Tcl_Alloc #define M

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Zoran Vasiljevic
On 19.12.2006, at 15:57, Vlad Seryakov wrote: Zoran, can you test it on Solaris and OSX so we'd know that is not Linux related problem. I have a Tcl library compiled with nedmalloc and when I link against it and make #define MemAlloc Tcl_Alloc #define MemFree Tcl_Free it runs fine. Shol

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Vlad Seryakov
I tried nedmalloc with LD_PRELOAD for my little test and it crashed vene before the start. Zoran, can you test it on Solaris and OSX so we'd know that is not Linux related problem. #include #include #include #include #include #include #define MemAlloc malloc #define MemFree free sta

Re: [naviserver-devel] Quest for malloc

2006-12-19 Thread Zoran Vasiljevic
On 19.12.2006, at 01:10, Stephen Deasey wrote: This program allocates memory in a worker thread and frees it in the main thread. If all free()'s put memory into a thread-local cache then you would expect this program to bloat, but it doesn't, so I guess it's not a problem (at least not on Fedor