Re: [twsocket] Very interesting issue with memory usage in ICSreverseproxy
I am starting to think that this is a CG bug. Reason: because when I maximize and minimize the debug form for the server, the memory usage falls from -no matter what MB- to 17MB! Anybody know a workaround for this leak? How should I search it? I am using BCB2007Pro. Regards, SZ On Wed, Jun 25, 2008 at 4:46 PM, S.Korotky <[EMAIL PROTECTED]> wrote: > SZ wrote: >> yes we hit the 2GB limit BUT WHY as we have 300MB >> memory usage for the rest of the program and the cache uses only 1GB. >> IOW, where does the 700MB go? > I'm afraid you should better know what memory overheads are produced in your > application than I or someone else. The actual memory consumption is shown > in the task manager, so if it says 2Gb this is 2Gb. > >> I think it is lost due to memory fragmentation >> (lack of FastMM doing what it's supposed to do). Has anybody seen such >> problem before? > The memory fragmentation is under your control to a certaing degree, depending > from memory allocation/reallocation/release policy (not only in the cache but > throughout the application). > > Anyway this seems a little bit offtopic for ICS. > > Best wishes, > Stanislav Korotky. > >> >> At a given time the proxy handles 1700 clients with 250Mbps and 40% cache >> hit rate with 10% CPU usage and 12 threads. So it's rather busy for >> memory than for CPU! >> >> Best Regards, >> >> SZ >> > -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Very interesting issue with memory usage in ICSreverseproxy
One last thing for tonight: I cannot use Memproof either: Create process : 3 LoadDLL : 7C80 ntdll.dll LoadDLL : 77E4 KERNEL32.dll LoadDLL : 76CF iphlpapi.dll LoadDLL : 77BA msvcrt.dll LoadDLL : 77F5 ADVAPI32.dll LoadDLL : 77C5 RPCRT4.dll LoadDLL : 76F5 Secur32.dll LoadDLL : 76B7 PSAPI.DLL LoadDLL : 7738 USER32.dll LoadDLL : 77C0 GDI32.dll LoadDLL : 71C0 WS2_32.dll LoadDLL : 71BF WS2HELP.dll LoadDLL : 7721 WININET.dll LoadDLL : 77DA SHLWAPI.dll LoadDLL : 761B CRYPT32.dll LoadDLL : 7619 MSASN1.dll LoadDLL : 77D0 OLEAUT32.dll LoadDLL : 7767 ole32.dll LoadDLL : 77B9 VERSION.dll LoadDLL : 71BB WSOCK32.dll LoadDLL : 7753 COMCTL32.dll LoadDLL : 7C8D SHELL32.dll Exception 8003 BREAKPOINT at 7C81A3E1 Create thread : Handle : 2036, ThreadID 4248 LoadDLL : 7742 COMCTL32.dll Exception C005 ACCESS_VIOLATION at 005DB0F5 Exception stack trace 005DB0F5 - in - in Exception C005 ACCESS_VIOLATION at 005DB0F5 Exception stack trace 005DB0F5 - in - in Thread Ended : 4248 Process Ended : 5 This is an optimized debug exe which runs well when standalone (except cache). I wonder what the heck it means! Regards, On Wed, Jun 25, 2008 at 5:01 PM, Fastream Technologies <[EMAIL PROTECTED]> wrote: > Hello, > > On Wed, Jun 25, 2008 at 4:46 PM, S.Korotky <[EMAIL PROTECTED]> wrote: >> >> SZ wrote: >> > yes we hit the 2GB limit BUT WHY as we have 300MB >> > memory usage for the rest of the program and the cache uses only 1GB. >> > IOW, where does the 700MB go? >> I'm afraid you should better know what memory overheads are produced in your >> application than I or someone else. The actual memory consumption is shown >> in the task manager, so if it says 2Gb this is 2Gb. > > In our cache design, we have one TMemoryStream per cached URL object > and it also has a critical section. So knowing the "memory overhead" > is not sufficient in our case... > >> >> > I think it is lost due to memory fragmentation >> > (lack of FastMM doing what it's supposed to do). Has anybody seen such >> > problem before? >> The memory fragmentation is under your control to a certaing degree, >> depending >> from memory allocation/reallocation/release policy (not only in the cache but >> throughout the application). > > In the code, the constructor for each URL cache object constructs the > memorystream and the cs. How else could it be?? > > >> >> Anyway this seems a little bit offtopic for ICS. > > A bit yes. But I think it will be valuable info for other 24/7 server authors. > > Thanks a lot, > > SZ > >> Best wishes, >> Stanislav Korotky. >> >> > >> > At a given time the proxy handles 1700 clients with 250Mbps and 40% cache >> > hit rate with 10% CPU usage and 12 threads. So it's rather busy for >> > memory than for CPU! >> > >> > Best Regards, >> > >> > SZ >> > > -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Very interesting issue with memory usage in ICSreverseproxy
Hello, On Wed, Jun 25, 2008 at 4:46 PM, S.Korotky <[EMAIL PROTECTED]> wrote: > > SZ wrote: > > yes we hit the 2GB limit BUT WHY as we have 300MB > > memory usage for the rest of the program and the cache uses only 1GB. > > IOW, where does the 700MB go? > I'm afraid you should better know what memory overheads are produced in your > application than I or someone else. The actual memory consumption is shown > in the task manager, so if it says 2Gb this is 2Gb. In our cache design, we have one TMemoryStream per cached URL object and it also has a critical section. So knowing the "memory overhead" is not sufficient in our case... > > > I think it is lost due to memory fragmentation > > (lack of FastMM doing what it's supposed to do). Has anybody seen such > > problem before? > The memory fragmentation is under your control to a certaing degree, depending > from memory allocation/reallocation/release policy (not only in the cache but > throughout the application). In the code, the constructor for each URL cache object constructs the memorystream and the cs. How else could it be?? > > Anyway this seems a little bit offtopic for ICS. A bit yes. But I think it will be valuable info for other 24/7 server authors. Thanks a lot, SZ > Best wishes, > Stanislav Korotky. > > > > > At a given time the proxy handles 1700 clients with 250Mbps and 40% cache > > hit rate with 10% CPU usage and 12 threads. So it's rather busy for > > memory than for CPU! > > > > Best Regards, > > > > SZ > > -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Very interesting issue with memory usage in ICSreverseproxy
SZ wrote: > yes we hit the 2GB limit BUT WHY as we have 300MB > memory usage for the rest of the program and the cache uses only 1GB. > IOW, where does the 700MB go? I'm afraid you should better know what memory overheads are produced in your application than I or someone else. The actual memory consumption is shown in the task manager, so if it says 2Gb this is 2Gb. > I think it is lost due to memory fragmentation > (lack of FastMM doing what it's supposed to do). Has anybody seen such > problem before? The memory fragmentation is under your control to a certaing degree, depending from memory allocation/reallocation/release policy (not only in the cache but throughout the application). Anyway this seems a little bit offtopic for ICS. Best wishes, Stanislav Korotky. > > At a given time the proxy handles 1700 clients with 250Mbps and 40% cache > hit rate with 10% CPU usage and 12 threads. So it's rather busy for > memory than for CPU! > > Best Regards, > > SZ > -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
