First, if you haven't already, you should checkout the performance info from
the .NET team [1]. Specifically, this article [2] on GC performance may be
valuable. Without understanding how the GC works it is relatively easy to
end up with code that it spending a lot of its time in GC. How high does
the time spent in GC get? It's not uncommon to have this around 10-15% for
moderately-well behaving applications. It might be valuable to monitor the
number of collections for the various generations. If you've having a lot
of Gen 2 collections then that may be what's taking all the time (they are
expensive). You might want to consider running the CLR profiler, or using
SOS to get a snapshot of your heap usage.
Is the Gen 2 heap size about what you'd expect (it seems pretty big compared
to the others)? I.e. are you hanging onto a lot of large objects or
something? If you're constantly allocating and freeing a lot of large
objects (which always go into gen 2) performance will suffer significantly -
consider an object pool (flyweight design pattern).
Is it possible you're just making requests faster than they can be
processed? I.e. if it takes 2 minutes to process a request on an unloaded
box, then firing one request a minute at it will cause the response time to
increase to infinity.
Hope this is helpful. Performance tuning in large .NET applications seems
to be much more of an art than a science, and I think we're still ALL
learning a lot about how to do it effectively (including Microsoft).
Rick
[1] http://www.gotdotnet.com/team/clr/about_clr_performance.aspx
[2] http://msdn.microsoft.com/library/en-us/dndotnet/html/dotnetgcbasics.asp
----- Original Message -----
From: "Bert Roos" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Tuesday, January 27, 2004 3:12 AM
Subject: Garbage collection problem
> Hi,
>
> We're having a strange problem that seems to be related to garbage
collection. We're performing load testing through a load generator tool
(LoadRunner). At a certain load level, the response times show strange and
heavy fluctuations. Request handling times go up by a factor 10 over an
extended period of time (several minutes, up to 10 minutes). Analysis of
performance counters gave some interesting results:
>
> * Context switches go up heavily
> * Contention rate/sec (.NET CLR LocksAndThreads) goes up heavily
> * Time spent in GC (.NET CLR Memory) goes up heavily
> * Gen 0 heap size (.NET CLR Memory) falls back from fluctuating
around 5MB to a flat 1MB
> * Gen 1 heap size (.NET CLR Memory) falls back from fluctuating
around 850KB to nearly nearly flat 350KB
> * Gen 2 heap size (.NET CLR Memory) keeps fluctuating around 29MB.
At one point, it goes up a few MBs in but this might be unrelated.
>
> The test is running on a 4 processor server with 1.5GB RAM.
>
> According to the documentation, the heap size for generation 0 is the
upper limit on memory allocated in generation 0. When this limit is reached,
garbage collection on generation 0 occurs. The garbage collector tunes this
value.
>
> Based on this and the numbers above, I come to the conclusion that the
context switches are caused by garbage collection. For some to me unknown
reason, the GC lowers its generation 0 heap size to a bare minimum. Due to
that, it must collect like crazy, practically killing the system.
>
> Can someone explain why this happens and how we can prevent it?
>
> Thanks, Bert Roos
>
>
> ===================================
> This list is hosted by DevelopMentor� http://www.develop.com
> Some .NET courses you may be interested in:
>
> NEW! Guerrilla ASP.NET, 26 Jan 2004, in Los Angeles
> http://www.develop.com/courses/gaspdotnetls
>
> View archives and manage your subscription(s) at
http://discuss.develop.com
>
===================================
This list is hosted by DevelopMentor� http://www.develop.com
Some .NET courses you may be interested in:
NEW! Guerrilla ASP.NET, 26 Jan 2004, in Los Angeles
http://www.develop.com/courses/gaspdotnetls
View archives and manage your subscription(s) at http://discuss.develop.com
