Re: [ADVANCED-DOTNET] windows service memory footprint
Courtney,
This issue has been discussed previously and, as Ian pointed out, the likely
problem is not that the runtime holds a lot of live objects, but that free
memory is not returned to the underlying OS.
You might want to use the following trick:
public class MemoryManagement
{
private MemoryManagement() {
}
[System.Runtime.InteropServices.DllImport (kernel32.dll)]
private extern static int SetProcessWorkingSetSize (IntPtr
hProcess, int dwMinimumWorkingSetSize, int dwMaximumWorkingSetSize);
public static void SwapOutProcess() {
GC.Collect();
GC.WaitForPendingFinalizers();
if (Environment.OSVersion.Platform ==
PlatformID.Win32NT)
SetProcessWorkingSetSize(Process.GetCurrentProcess().Handle, -1, -1);
}
}
Works for me. It will basically 'flush out' anything unused from the RAM
your process is holding into oblivion; this is the same effect you would
notice with a GUI app when you minimize the application's main window.
Don't call it too often, though; once at the end of your service startup
(after you've initialized everything) and once after your nightly process
has completed would probably be more than enough.
And, tell me if it works (you can even do it by private email if you wish)
because I'm interested in any implications; I've used it extensively in
WinForms apps, but not with a service or IIS-hosted application.
Kamen
-Original Message-
From: Moderated discussion of advanced .NET topics.
[mailto:[EMAIL PROTECTED] On Behalf Of Courtney Smith
Sent: 31 2003 . 16:33
To: [EMAIL PROTECTED]
Subject: [ADVANCED-DOTNET] windows service memory footprint
Thanks to everyone for the replies. I've done
everything short of calling the garbage collector
which has been suggested against both in print and
verbally. As for the SQL solution Russ suggest below,
I think my collection of SQL stored procs that gets
called to create my reports its more complicated than
my SQL abilities to create on my own into one massive
stored procI also need to make these into html
emails, nicely formatted...something System.web.mail
does very nicely for me to send out to from anywhere
between 0 - 300 people per day. This isn't spam by
any means. Its internal to our organization. :)
It appears the memory usage depends on the quantity of
the emails sent out that day but I'm destorying
(setting to nothing) all of the DataSets and
DataReaders used. As mentioned in a pervious post, it
likes 7MB just to exists as a service in the Service
Manager running a 24 hour timer.
Courtney
Date:Wed, 30 Jul 2003 09:37:38 -0700
From:Russell McClure
[EMAIL PROTECTED]
Subject: Re: windows service memory footprint
Courtney,
Just a thought (I'm assuming you are using SQL
Server): you could actually
leverage the SQL Server Agent to do what you describe.
That way you
wouldn't even need your own service. All you would
need to do is to create
a Job under the SQL Server Agent and specify the times
that you want it to
launch. Then add steps to the job that call the
stored procedures and send
emails etc. This would all be written in T-SQL.
Anyway, if this sounds interesting, let me know and I
can give you more
detail if you want.
Russ
=
Courtney Smith
[EMAIL PROTECTED]
Re: [ADVANCED-DOTNET] windows service memory footprint
Are you looking at the process virtual memory size or the working set size? These two will give quite a different picture of what your process is doing. (It's also sometimes a less than exact science interpreting this stuff, since measuring a process's memory use is not always that clear cut - if a process causes certain pages from system DLLs to be loaded in, is that part of its working set? Presumably, but what if it's not the only process in the system using those pages?.. You tend to end up with a total 'memory in use' figure that's substantially larger than the amount of memory you have.) It is possible that your program really is sitting on that much memory in a way that the garbage collector can't do anything with. Alternatively, it's possible that the garbage collector found all your free memory, but elected not to return that memory to the OS when you called System.GC.Collect, because it saw that there was plenty of free memory in the system, so there was no point wasting CPU cycles to free up memory that (a) isn't needed by anything else and (b) might well be needed again by this process. One way to find out how much memory your process is really using is to use the performance monitor - it has some GC measurements that can be helpful. Or you can run it through the .NET Allocation Profiler: http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=3 6a3e666-6877-4c26-b62d-bfd7cb3154ac This will show you in detail how your program is using memory. It takes a little learning to use this tool effectively, but it's well worth the effort. -- Ian Griffiths DevelopMentor -Original Message- From: Paulo Jorge F. Sacramento [mailto:[EMAIL PROTECTED] I hope that's true. I've had the same problem with my services, which are written in C#. Mine are even worse. I have one that takes up 23 MB. It's using Remoting, AppDomains, Threads, XML, some Reflection also, and other things, but nothing that I'd think would occupy so much memory. The reason I'm not so sure about your explanation is that I've tried explicitly calling the GC (with System.GC() or something like that) and it didn't do much. If what you say is right, wouldn't it be expectable that an explicit call to the GC would release a probably big chunk of memory? By the way, I've also had a case with a much simpler service that occupied about 6 or 7 MB at the beggining of execution, but that after a while would occupy only a few KBs. It was sleeping most of the time and did a very rare periodical check. I guess the O.S. together with the GC take care of releasing all unneeded resources. This also suggests that .Net services have an heavy start-up penalty. Any thoughts on this? Paulo Sacramento On Wed, 30 Jul 2003, Griffiths, Ian wrote: .NET processes will tend to grow to what looks like an alarmingly large size unless there is a reason for them to shrink. The CLR does actually keep track of the amount of memory being used system-wide, and if other processes start to demand more memory, the CLR will trigger a garbage collect in your process, reducing the amount of memory that it is using. So the large-looking memory consumption isn't usually an issue. If anything else in the system starts to need that memory, the CLR will give it back. IDispose doesn't have any bearing on this. IDispose has nothing to do with memory allocation, it's just about other resources. The fact that your memory usage stays constant at 18MB indicates that you're not leaking resources. If it was forever creeping upwards, then that would be something to worry about. But 18MB is fairly normal. -- We live in a society exquisitely dependent on science and technology, in which hardly anyone knows anything about science and technology. Carl Sagan
Re: [ADVANCED-DOTNET] windows service memory footprint
The term for these 'levels' is generations. In fact there are 3 generations, numbered 0, 1, and 2. Objects in generation zero are those which were allocated after the most recent garbage collection to have completed. What you say is true - objects that are in generation 0 that are still in use at the time a GC happens get promoted to generation 1. And even if they subsequently fall out of use, the memory won't necessarily be reclaimed at the next garbage collect. The system only bothers to collect objects from generation 1 relatively infrequently. (It depends on the memory usage characteristics of the application, but it would be fairly common to only see 1 generation 1 collection for every 10 generation 0 collections.) If a generation 1 collection happens, any objects that were already in generation 1 and are still in use get promoted to generation 2. Generation 2 collections happen even less often. So if your object is still in use when the first garbage collection happens, expect to wait for something on the order of 10 GCs to occur between the object falling out of use and the memory it occupies being reclaimed. And if it survives long enough to make it into generation 2, you might conceivably see as many as 100 garbage collections occurring before the memory is reclaimed. However, having gone through all that I don't think that's what's happening here. I believe that calling System.GC.Collect() forces a full GC, i.e. it collects all three levels now. So I don't think this is the generational GC in action. The thing that I'm not sure about is this: does the memory manager in .NET necessarily return all of the memory collected back to the OS? I'm guessing not, since GCs usually happen as a result of the program needing more memory than was available in the heap. So it needs to keep at least some memory back for itself in order to be able to allocate new objects. I'm just not sure how much it leaves. (Nor do I know what mechanisms it uses to allocate memory - there are lots of different ways to signal your requirements to the OS memory manager that it might use...) -- Ian Griffiths DevelopMentor -Original Message- From: Brandon Manchester [mailto:[EMAIL PROTECTED] I may be misstating this so if I am please correct me. There are 2 levels that objects are kept at in regards to garbage collection. The actual name of these levels evades me at this moment so I will just refer to them as level 1 and 2. With that said, the first level is where all objects are at until the GC runs. When the GC runs it checks to see if there are any references to the object in question. If there is a reference then it will not destroy and collect the object(s) at that time, what it does it pushes that object to the 2nd level. When object(s) are in the 2nd level the GC does not look at them until memory becomes an issue for other processes. I believe this is what Ian was talking about. Even though you explicitly call the GC this same process takes place. So there is not guarantee as to when an object will be taken out of memory. As I mentioned above I may be mistaken and if so please correct me =) Hope this helps. Brandon -Original Message- From: Paulo Jorge F. Sacramento [mailto:[EMAIL PROTECTED] Sent: Wednesday, July 30, 2003 11:01 AM To: [EMAIL PROTECTED] Subject: Re: [ADVANCED-DOTNET] windows service memory footprint I hope that's true. I've had the same problem with my services, which are written in C#. Mine are even worse. I have one that takes up 23 MB. It's using Remoting, AppDomains, Threads, XML, some Reflection also, and other things, but nothing that I'd think would occupy so much memory. The reason I'm not so sure about your explanation is that I've tried explicitly calling the GC (with System.GC() or something like that) and it didn't do much. If what you say is right, wouldn't it be expectable that an explicit call to the GC would release a probably big chunk of memory? By the way, I've also had a case with a much simpler service that occupied about 6 or 7 MB at the beggining of execution, but that after a while would occupy only a few KBs. It was sleeping most of the time and did a very rare periodical check. I guess the O.S. together with the GC take care of releasing all unneeded resources. This also suggests that .Net services have an heavy start-up penalty. Any thoughts on this? Paulo Sacramento On Wed, 30 Jul 2003, Griffiths, Ian wrote: .NET processes will tend to grow to what looks like an alarmingly large size unless there is a reason for them to shrink. The CLR does actually keep track of the amount of memory being used system-wide, and if other processes start to demand more memory, the CLR will trigger a garbage collect in your process, reducing the amount of memory that it is using. So the large-looking memory consumption isn't usually an issue. If anything else in the system starts to need that memory, the CLR will give it back
Re: [ADVANCED-DOTNET] windows service memory footprint
.NET processes will tend to grow to what looks like an alarmingly large size unless there is a reason for them to shrink. The CLR does actually keep track of the amount of memory being used system-wide, and if other processes start to demand more memory, the CLR will trigger a garbage collect in your process, reducing the amount of memory that it is using. So the large-looking memory consumption isn't usually an issue. If anything else in the system starts to need that memory, the CLR will give it back. IDispose doesn't have any bearing on this. IDispose has nothing to do with memory allocation, it's just about other resources. The fact that your memory usage stays constant at 18MB indicates that you're not leaking resources. If it was forever creeping upwards, then that would be something to worry about. But 18MB is fairly normal. -- Ian Griffiths DevelopMentor -Original Message- From: Courtney Smith [mailto:[EMAIL PROTECTED] Hello, I'm taking a stab at windows services for the first time. My windows service is written in vb.net (not my lang of choice but I have to go with the flow of the department). It is running a once a day timer that calls several stored procs and depending on the size of the returned values(s) sends out emails. Now for my questions. Can I expect Windows Services in .Net to have a considerable memory foot print? I check and rechecked the closure of my only dataset and datareader objects. I have also set them to nothing when I have finished using them. This service is only called once a day. When it first started it uses a woping 7MB of RAM...then on first execution it uses 18MB and this stays constant execution after execution. I monitored the size hoping the garbage collector might help me out but to noavail. I do not implement IDispose if that helps.
Re: [ADVANCED-DOTNET] windows service memory footprint
I hope that's true. I've had the same problem with my services, which are written in C#. Mine are even worse. I have one that takes up 23 MB. It's using Remoting, AppDomains, Threads, XML, some Reflection also, and other things, but nothing that I'd think would occupy so much memory. The reason I'm not so sure about your explanation is that I've tried explicitly calling the GC (with System.GC() or something like that) and it didn't do much. If what you say is right, wouldn't it be expectable that an explicit call to the GC would release a probably big chunk of memory? By the way, I've also had a case with a much simpler service that occupied about 6 or 7 MB at the beggining of execution, but that after a while would occupy only a few KBs. It was sleeping most of the time and did a very rare periodical check. I guess the O.S. together with the GC take care of releasing all unneeded resources. This also suggests that .Net services have an heavy start-up penalty. Any thoughts on this? Paulo Sacramento On Wed, 30 Jul 2003, Griffiths, Ian wrote: .NET processes will tend to grow to what looks like an alarmingly large size unless there is a reason for them to shrink. The CLR does actually keep track of the amount of memory being used system-wide, and if other processes start to demand more memory, the CLR will trigger a garbage collect in your process, reducing the amount of memory that it is using. So the large-looking memory consumption isn't usually an issue. If anything else in the system starts to need that memory, the CLR will give it back. IDispose doesn't have any bearing on this. IDispose has nothing to do with memory allocation, it's just about other resources. The fact that your memory usage stays constant at 18MB indicates that you're not leaking resources. If it was forever creeping upwards, then that would be something to worry about. But 18MB is fairly normal. -- We live in a society exquisitely dependent on science and technology, in which hardly anyone knows anything about science and technology. Carl Sagan
