Re: [Mono-dev] Need help tracking this bug...
On 03/30/07 Alan McGovern wrote: type=System.MonoType 2826 542.8M 201403.8 0.6 Half a gigabyte of System.MonoType's? What causes these to be allocated, and what can i do to stop this? That data must be incorrect. MonoType is fixed-size (16 bytes on 32 bit systems), so the data is inconsistent. Having 2826 instances is reasonable (though it's a hint that you or somebody else is doing GeTypes() on the assemblies and that should be avoided). The 543 MB for MonoType is totally bogus. lupus -- - [EMAIL PROTECTED] debian/rules [EMAIL PROTECTED] Monkeys do it better ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hi, I just compiled the latest SVN head, and once again monotorrent works perfectly (well, as perfectly as to be expected ;) ). No more crazy memory usage, which means it was a mono bug after all. I just wish i knew what exactly had gone wrong inside mono. For the amount of time i spent trying to figure out what it was, i feel cheated that someone else fixed it before i could ;) Anyway, thanks for the help joe, i couldn't have gotten as far as i did without you. Alan. On 3/31/07, Alan McGovern [EMAIL PROTECTED] wrote: Ok, i did a quick bit of testing on my code. Firstly, the 54 instances of SocketAsyncResult do make sense. Typically there will *always* be a pending BeginReceive on every socket i have opened. Therefore 54 instances of SocketAsyncResult would correspond to 54 open connections, which is pretty standard stuff. Every call to socket.BeginXXX or socket.EndXXX goes through a TCPConnection class, so i added a count there so i could see how many calls to each BeginXXX and EndXXX method i actually make. The results were pretty much as expected. There was always a difference between BeginReceive and EndReceive pretty much equal to the number of sockets i have open. The difference between BeginSend and EndSend was always close enough to zero, which would also make sense as most messages i'm sending would only be a few bytes in size. The difference between the BeginConnects and EndConnects is always 5. This is right as i've set it so i only connect to at most 5 people at any one instant. So i can rule out a bug in my code that's causing millions of BeginXXX's without corresponding EndXXX's. There'd typically be between 80 and 120 Begin/EndReceive calls a second, which isn't that much really. A similar amount for Begin/EndSend. On 3/31/07, Alan McGovern [EMAIL PROTECTED] wrote: Hi, That leaves me with the question of how the hell a SocketAsyncResult is 10 megs in size! The size of the largest single object in my entire code is a 16kB byte[] buffer. If each SocketAsyncResult is 10 megabytes in size, i have to question the internal workings of mono, as i know from profiling my own code does *not* create objects anywhere near that size. I'm going to do a bit of profiling to count how many socket BeginXXX calls are made from my own code as compared to the EndXXX calls to see how they match up. Is there any way of finding out what exactly is inside those SocketAsyncResults that is 10 megs in size? I can verify that the exact same code running under Mono 1.2 and earlier does *not* exhibit the same behavior, everything works fine. I only came across this bug after updating my mono installation to 1.2.3. This is why i think it's a mono bug, however i can't reproduce the problem in the form of an NUnit test. Thanks again, Alan. On 3/30/07, Joe Shaw [EMAIL PROTECTED] wrote: Hi again, On 3/30/07, Joe Shaw [EMAIL PROTECTED] wrote: That huge object array, in turn, is referenced by System.Collections.Queue - System.Net.Socket.Socket , specifically the readQ member. So basically it means that the readQ member in System.Net.Socket.Socket is a huge Queue, which internally has an object array, which apparently has millions of SocketAsyncResult objects inside. So how those are being allocated? These objects are created a lot[...] I got a little ahead of myself here: I'm obviously looking at the code for System.Net.Sockets.Socket here. I don't know much about the Socket class and how the async IO works, but it boils down to the fact that BeginReceive() is being called probably millions of times, but it doesn't look like Complete() is being called enough (or possibly at all) to balance the load. I actually noticed something else: The object array in question has an average size of 10.6 megs, but it only holds 56 references to SocketAsyncResult at the time of this snapshot. So this seems to indicate to me that the enqueues and dequeues do ultimately match up, but that the allocation pattern is bad and probably not interspersed. That is, enqueue, enqueue, dequeue, dequeue, enqueue, enqueue, etc. would mean that an array could be as small as 2 items and still work for N items, assuming a 1:1 match.. But if the pattern is instead, enqueue, enqueue, enqueue, enqueue, dequeue, dequeue, etc. then it would have to be at least N. Now pretend N is a million. :) Joe ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hi Alan, On 3/29/07, Alan McGovern [EMAIL PROTECTED] wrote: I ran it again, and came up with this (startling ;) ) figures for allocations: 1) type=System.Runtime.Remoti...ing.MonoMethodMessage 4185 1024.1M 256603.6 0.5 System.Threading.ThreadPool:QueueUserWorkItem System.Timers.Timer:StartTimer What the hell? 1 gig of allocations from a System.Timers.Timer? Whats going on here!? Yep, that's your problem right there. ;) Also, if you look at the summary you posted, you'll notice that in the course of 1 second, you have 9 resizes going from a size of 2.8 megs to 322.6 megs. On the surface it looks like a Mono bug, or possibly a bug in the amount of data you're processing through remoting. As Miguel suggested, using heap-shot at this point will be helpful in finding out what objects are allocating these and hopefully why. Joe ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hi, On 3/30/07, Miguel de Icaza [EMAIL PROTECTED] wrote: One question: how do I enable the four different profiling modes for HeapBuddy? The summary mode is the default. For the others, you just provide the relevant word on the command line. history, types, backtraces. You can also turn off the autoabbreviation by saying something like full names and change the sorting with sort by age, for example. Joe ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hi, Yep, that's your problem right there. ;) Also, if you look at the summary you posted, you'll notice that in the course of 1 second, you have 9 resizes going from a size of 2.8 megs to 322.6 megs. Yup, i had noticed that ;) There is something fishy, but i can't find out what it is. I've run heap-shot and attached and emailed the results in already. Hopefully they'll make sense to someone. By the looks of it, there's some kind of problem internal to mono which is resulting in this rapid memory usage as on MS.NET this doesn't happen. There could easily be a bug in my own code and MS.NET just handles it better, but i have no idea what is causing the problem to start off with, so i can't fix it. Im not using remoting as such, I assume that internally AsyncSockets make use of the remoting framework. I do make heavy use of those, but i'm using them in the way they're supposed to be used. Any further advice would be brilliant. Alan. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hi Alan, If you can run your application on Windows, check this tool which can help to find where's located memory leak: http://www.microsoft.com/downloads/details.aspx?amp;amp;displaylang=enfamilyid=9bfa49bc-376b-4a54-95aa-73c9156706e7displaylang=en. This guide can also help: http://support.microsoft.com/?scid=kb%3Ben-us%3B919790x=11y=15. From your e-mail, it seems to be those AsyncSockets not disposing as expected. Cheers, 2007/3/30, Alan McGovern [EMAIL PROTECTED]: Hi, Yep, that's your problem right there. ;) Also, if you look at the summary you posted, you'll notice that in the course of 1 second, you have 9 resizes going from a size of 2.8 megs to 322.6 megs. Yup, i had noticed that ;) There is something fishy, but i can't find out what it is. I've run heap-shot and attached and emailed the results in already. Hopefully they'll make sense to someone. By the looks of it, there's some kind of problem internal to mono which is resulting in this rapid memory usage as on MS.NET this doesn't happen. There could easily be a bug in my own code and MS.NET just handles it better, but i have no idea what is causing the problem to start off with, so i can't fix it. Im not using remoting as such, I assume that internally AsyncSockets make use of the remoting framework. I do make heavy use of those, but i'm using them in the way they're supposed to be used. Any further advice would be brilliant. Alan. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list -- Marcos - http://www.youcannoteatbits.org ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Just a quick update, i think my email bounced due to the attached heap-shot data. I'm now hosting it here: http://www.megaupload.com/?d=QOJKRWT2 If theres problems with that, let me know. Alan. On 3/30/07, Marcos Cobeña Morián [EMAIL PROTECTED] wrote: Hi Alan, If you can run your application on Windows, check this tool which can help to find where's located memory leak: http://www.microsoft.com/downloads/details.aspx?amp;amp;displaylang=enfamilyid=9bfa49bc-376b-4a54-95aa-73c9156706e7displaylang=en . This guide can also help: http://support.microsoft.com/?scid=kb%3Ben-us%3B919790x=11y=15. From your e-mail, it seems to be those AsyncSockets not disposing as expected. Cheers, 2007/3/30, Alan McGovern [EMAIL PROTECTED]: Hi, Yep, that's your problem right there. ;) Also, if you look at the summary you posted, you'll notice that in the course of 1 second, you have 9 resizes going from a size of 2.8 megs to 322.6 megs. Yup, i had noticed that ;) There is something fishy, but i can't find out what it is. I've run heap-shot and attached and emailed the results in already. Hopefully they'll make sense to someone. By the looks of it, there's some kind of problem internal to mono which is resulting in this rapid memory usage as on MS.NET this doesn't happen. There could easily be a bug in my own code and MS.NET just handles it better, but i have no idea what is causing the problem to start off with, so i can't fix it. Im not using remoting as such, I assume that internally AsyncSockets make use of the remoting framework. I do make heavy use of those, but i'm using them in the way they're supposed to be used. Any further advice would be brilliant. Alan. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list -- Marcos - http://www.youcannoteatbits.org ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
From your e-mail, it seems to be those AsyncSockets not disposing as expected Even if i wasn't disposing of them at all, i connect to at most 5 people a second, so that's at most 5 sockets being created a second. There's no way that they would lead to such a huge allocation rate. Also, i can monitor the number of ports/handles i have open and when memory usage rockets, they remain stable. So that should indicate that it isn't run-away creation of sockets. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hi, I gave that a shot, but i can't actually make use of it :( The problem is that the memory usage goes from 20 megabytes to 2 gigabytes in the space of 2 seconds. I can't get a snapshot of anywhere in between and once memory usage rockets up there, i can't take a snapshot using the tool. So once again, i'm at a loss. Alan On 3/30/07, Marcos Cobeña Morián [EMAIL PROTECTED] wrote: Hi Alan, If you can run your application on Windows, check this tool which can help to find where's located memory leak: http://www.microsoft.com/downloads/details.aspx?amp;amp;displaylang=enfamilyid=9bfa49bc-376b-4a54-95aa-73c9156706e7displaylang=en . This guide can also help: http://support.microsoft.com/?scid=kb%3Ben-us%3B919790x=11y=15. From your e-mail, it seems to be those AsyncSockets not disposing as expected. Cheers, 2007/3/30, Alan McGovern [EMAIL PROTECTED]: Hi, Yep, that's your problem right there. ;) Also, if you look at the summary you posted, you'll notice that in the course of 1 second, you have 9 resizes going from a size of 2.8 megs to 322.6 megs. Yup, i had noticed that ;) There is something fishy, but i can't find out what it is. I've run heap-shot and attached and emailed the results in already. Hopefully they'll make sense to someone. By the looks of it, there's some kind of problem internal to mono which is resulting in this rapid memory usage as on MS.NET this doesn't happen. There could easily be a bug in my own code and MS.NET just handles it better, but i have no idea what is causing the problem to start off with, so i can't fix it. Im not using remoting as such, I assume that internally AsyncSockets make use of the remoting framework. I do make heavy use of those, but i'm using them in the way they're supposed to be used. Any further advice would be brilliant. Alan. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list -- Marcos - http://www.youcannoteatbits.org ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hey, On 3/30/07, Alan McGovern [EMAIL PROTECTED] wrote: Yup, i had noticed that ;) There is something fishy, but i can't find out what it is. I've run heap-shot and attached and emailed the results in already. Hopefully they'll make sense to someone. By the looks of it, there's some kind of problem internal to mono which is resulting in this rapid memory usage as on MS.NET this doesn't happen. There could easily be a bug in my own code and MS.NET just handles it better, but i have no idea what is causing the problem to start off with, so i can't fix it. I just took a look at your heap-shot files, and it's a little odd. Specifically in Second Run/outfile_1.omap: object[] is the obvious problem. 367 instances, 490 megs, average size: 1.3 megs. You have to drill down in the inverse references to find specifically the big consumer: object[] - System.Collections.Queue - System.Net.Sockets.Socket - System.Net.Sockets.Socket/SocketAsyncResult - object[]. It has 35 instances taking 373 megs, with an average of 10.6 megs each. That huge object array, in turn, is referenced by System.Collections.Queue - System.Net.Socket.Socket, specifically the readQ member. So basically it means that the readQ member in System.Net.Socket.Socket is a huge Queue, which internally has an object array, which apparently has millions of SocketAsyncResult objects inside. So how those are being allocated? These objects are created a lot, but the ones that create them and enqueue them into readQ are BeginReceive() and BeginReceiveFrom(). That queue is dequeued inside the Complete() method. I don't know much about the Socket class and how the async IO works, but it boils down to the fact that BeginReceive() is being called probably millions of times, but it doesn't look like Complete() is being called enough (or possibly at all) to balance the load. Hope this helps, Joe ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hi again, On 3/30/07, Joe Shaw [EMAIL PROTECTED] wrote: That huge object array, in turn, is referenced by System.Collections.Queue - System.Net.Socket.Socket, specifically the readQ member. So basically it means that the readQ member in System.Net.Socket.Socket is a huge Queue, which internally has an object array, which apparently has millions of SocketAsyncResult objects inside. So how those are being allocated? These objects are created a lot[...] I got a little ahead of myself here: I'm obviously looking at the code for System.Net.Sockets.Socket here. I don't know much about the Socket class and how the async IO works, but it boils down to the fact that BeginReceive() is being called probably millions of times, but it doesn't look like Complete() is being called enough (or possibly at all) to balance the load. I actually noticed something else: The object array in question has an average size of 10.6 megs, but it only holds 56 references to SocketAsyncResult at the time of this snapshot. So this seems to indicate to me that the enqueues and dequeues do ultimately match up, but that the allocation pattern is bad and probably not interspersed. That is, enqueue, enqueue, dequeue, dequeue, enqueue, enqueue, etc. would mean that an array could be as small as 2 items and still work for N items, assuming a 1:1 match.. But if the pattern is instead, enqueue, enqueue, enqueue, enqueue, dequeue, dequeue, etc. then it would have to be at least N. Now pretend N is a million. :) Joe ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hi, That leaves me with the question of how the hell a SocketAsyncResult is 10 megs in size! The size of the largest single object in my entire code is a 16kB byte[] buffer. If each SocketAsyncResult is 10 megabytes in size, i have to question the internal workings of mono, as i know from profiling my own code does *not* create objects anywhere near that size. I'm going to do a bit of profiling to count how many socket BeginXXX calls are made from my own code as compared to the EndXXX calls to see how they match up. Is there any way of finding out what exactly is inside those SocketAsyncResults that is 10 megs in size? I can verify that the exact same code running under Mono 1.2 and earlier does *not* exhibit the same behavior, everything works fine. I only came across this bug after updating my mono installation to 1.2.3. This is why i think it's a mono bug, however i can't reproduce the problem in the form of an NUnit test. Thanks again, Alan. On 3/30/07, Joe Shaw [EMAIL PROTECTED] wrote: Hi again, On 3/30/07, Joe Shaw [EMAIL PROTECTED] wrote: That huge object array, in turn, is referenced by System.Collections.Queue - System.Net.Socket.Socket, specifically the readQ member. So basically it means that the readQ member in System.Net.Socket.Socket is a huge Queue, which internally has an object array, which apparently has millions of SocketAsyncResult objects inside. So how those are being allocated? These objects are created a lot[...] I got a little ahead of myself here: I'm obviously looking at the code for System.Net.Sockets.Socket here. I don't know much about the Socket class and how the async IO works, but it boils down to the fact that BeginReceive() is being called probably millions of times, but it doesn't look like Complete() is being called enough (or possibly at all) to balance the load. I actually noticed something else: The object array in question has an average size of 10.6 megs, but it only holds 56 references to SocketAsyncResult at the time of this snapshot. So this seems to indicate to me that the enqueues and dequeues do ultimately match up, but that the allocation pattern is bad and probably not interspersed. That is, enqueue, enqueue, dequeue, dequeue, enqueue, enqueue, etc. would mean that an array could be as small as 2 items and still work for N items, assuming a 1:1 match.. But if the pattern is instead, enqueue, enqueue, enqueue, enqueue, dequeue, dequeue, etc. then it would have to be at least N. Now pretend N is a million. :) Joe ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hi, On 3/30/07, Alan McGovern [EMAIL PROTECTED] wrote: That leaves me with the question of how the hell a SocketAsyncResult is 10 megs in size! It's not the SocketAsyncResult itself that's 10 megs in size, it's the object array that holds them that is 10 megs in size. I'm going to do a bit of profiling to count how many socket BeginXXX calls are made from my own code as compared to the EndXXX calls to see how they match up. I think this is a good idea, and if that doesn't work out, I would use Mono's tracing facility to see how often the methods I mentioned in my last email were being called and in what order. And if that doesn't work, instrument the class libs themselves. I can verify that the exact same code running under Mono 1.2 and earlier does *not* exhibit the same behavior, everything works fine. I only came across this bug after updating my mono installation to 1.2.3. This is why i think it's a mono bug, however i can't reproduce the problem in the form of an NUnit test. It's probably in the sockets code, so take a diff of System.Net.Sockets.Socket between 1.2 and 1.2.3 and see what, if anything changed. If you can narrow it down to a specific change between the releases, that will hopefully make it easier to write a test case. Joe ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Ok, i did a quick bit of testing on my code. Firstly, the 54 instances of SocketAsyncResult do make sense. Typically there will *always* be a pending BeginReceive on every socket i have opened. Therefore 54 instances of SocketAsyncResult would correspond to 54 open connections, which is pretty standard stuff. Every call to socket.BeginXXX or socket.EndXXX goes through a TCPConnection class, so i added a count there so i could see how many calls to each BeginXXX and EndXXX method i actually make. The results were pretty much as expected. There was always a difference between BeginReceive and EndReceive pretty much equal to the number of sockets i have open. The difference between BeginSend and EndSend was always close enough to zero, which would also make sense as most messages i'm sending would only be a few bytes in size. The difference between the BeginConnects and EndConnects is always 5. This is right as i've set it so i only connect to at most 5 people at any one instant. So i can rule out a bug in my code that's causing millions of BeginXXX's without corresponding EndXXX's. There'd typically be between 80 and 120 Begin/EndReceive calls a second, which isn't that much really. A similar amount for Begin/EndSend. On 3/31/07, Alan McGovern [EMAIL PROTECTED] wrote: Hi, That leaves me with the question of how the hell a SocketAsyncResult is 10 megs in size! The size of the largest single object in my entire code is a 16kB byte[] buffer. If each SocketAsyncResult is 10 megabytes in size, i have to question the internal workings of mono, as i know from profiling my own code does *not* create objects anywhere near that size. I'm going to do a bit of profiling to count how many socket BeginXXX calls are made from my own code as compared to the EndXXX calls to see how they match up. Is there any way of finding out what exactly is inside those SocketAsyncResults that is 10 megs in size? I can verify that the exact same code running under Mono 1.2 and earlier does *not* exhibit the same behavior, everything works fine. I only came across this bug after updating my mono installation to 1.2.3. This is why i think it's a mono bug, however i can't reproduce the problem in the form of an NUnit test. Thanks again, Alan. On 3/30/07, Joe Shaw [EMAIL PROTECTED] wrote: Hi again, On 3/30/07, Joe Shaw [EMAIL PROTECTED] wrote: That huge object array, in turn, is referenced by System.Collections.Queue - System.Net.Socket.Socket , specifically the readQ member. So basically it means that the readQ member in System.Net.Socket.Socket is a huge Queue, which internally has an object array, which apparently has millions of SocketAsyncResult objects inside. So how those are being allocated? These objects are created a lot[...] I got a little ahead of myself here: I'm obviously looking at the code for System.Net.Sockets.Socket here. I don't know much about the Socket class and how the async IO works, but it boils down to the fact that BeginReceive() is being called probably millions of times, but it doesn't look like Complete() is being called enough (or possibly at all) to balance the load. I actually noticed something else: The object array in question has an average size of 10.6 megs, but it only holds 56 references to SocketAsyncResult at the time of this snapshot. So this seems to indicate to me that the enqueues and dequeues do ultimately match up, but that the allocation pattern is bad and probably not interspersed. That is, enqueue, enqueue, dequeue, dequeue, enqueue, enqueue, etc. would mean that an array could be as small as 2 items and still work for N items, assuming a 1:1 match.. But if the pattern is instead, enqueue, enqueue, enqueue, enqueue, dequeue, dequeue, etc. then it would have to be at least N. Now pretend N is a million. :) Joe ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
[Mono-dev] Need help tracking this bug...
Hi, Basically, i have a major (ish) problem in monotorrent with Mono 1.2.1 and above. After spending a few hours backtracking through svn revisions, i've managed to place the origin of the problem to between revision 72227 and 72228 of monotorrent in the mono svn. To generate the diff just use: svn diff -r 72227:72228 svn://svn.myrealbox.com/source/trunk/bitsharp . I've also attached the diff. The changes in this diff are solely related to a refactor where i moved three seperate lists into one main PeerList class. Now, after careful examination of that diff, i just can't see anything in it that would cause the problems i'm seeing. Also, to further complicate matters, if i step back to the mono 1.2 release, everything appears to work perfectly. Now, the actual bug is that memory usage rockets to 100% after a very short running time. Typically 30 seconds is plenty of time for the bug to manifest on a well seeded torrent. I've seen this problem once before on completely different code (however i can't remember who i was helping track down the source, but it was a guy on #mono). His code was a crawler for another P2P network and after a similar amount of time, it would hit 100% memory usage aswell. I profiled his code using the mono profiler and eventually managed to get some usable output which pointed out that he was allocating 1's of instances of multi-cast delegates which were just eating memory. At the time, i put this down to bad programming or incorrect usage of async sockets. So, the thing is i can't track down what's causing the memory usage in monotorrent. I have no idea how to even start. I've tried using the built in profilers, but they require a graceful exit before they output their data, and unfortunately i can't get them a graceful exit ;) If anyone has any ideas on how i could track this problem, or if they need more information, please let me know. I'm going demented trying to figure out the problem ;) Also, for anyone trying to compile revision 72227 from svn, you will need the BufferedFileRead.cs and BufferedFileWrite.cs classes which were commited in rev 72228. I accidently left them out of a previous commit. Revision 72228 will compile fine (i hope) from a fresh checkout with no messing. Thanks, Alan. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hi Alan, On 3/29/07, Alan McGovern [EMAIL PROTECTED] wrote: So, the thing is i can't track down what's causing the memory usage in monotorrent. I have no idea how to even start. I've tried using the built in profilers, but they require a graceful exit before they output their data, and unfortunately i can't get them a graceful exit ;) I would use the two external memory profilers for this: heap-buddy and heap-shot. Both are available from Mono SVN and straightforward to build and use. heap-buddy is a summarizing profiler. It collects allocation statistics while the program is running and writes out the aggregate info to a file which you analyze with a command-line tool after it's finished running. It is robust against your app exiting ungracefully. :) The tool has 4 basic modes: summary, history, types and backtraces. Summary gives you high-level info: how many objects/bytes were allocated, the final size of the heap, the number of GCs, etc. History gives you a timeline of each heap resize and GC, telling you the sizes of the heap, the number of objects and their size, and how it changed over time. You can see, for instance, that if your heap resizes 5 times in a row without a GC all within 2 seconds that you have a run away allocation pattern. :) Types will show you all the value types that have been created, how many total instances, the total bytes, the average size of one instance, and the average age (ie, the number of GCs) it lived for. If you're dealing with runaway memory, this will tell you what it is. The average age is good for finding memory leaks. Backtraces is like types, except it's more granular. It shows you the same info as types, but it breaks the types up by the call trace that allocated them. This is good when you're trying to find out what is allocating all those damn strings. :) The other tool is heap-shot, which is a snapshotting profiler. When you use it, you send SIGPROF to your running process which causes a file to be written out with all the information about allocations at that point. Afterward you can use the GUI to see where all the memory has gone. The basic display mode shows you all the types, along with the number of instances and the number of bytes these use. This is pretty similar to heap-buddy's types mode, except that it shows you the info for a snapshot in time, rather than the aggregate over the whole run of the program. The beauty of this tool is in the Reverse references mode, which tells you what objects hold references to a given type. So, if you reverse the references for string, you might see that 100 instances of string are being referenced by 50 instances of System.IO.FileInfo, and are using 2k of memory. As you drill down, you might see reference leaks (like those damned static ArrayLists and Hashtables) that continually grow without bound. These two tools (especially heap-shot) have helped us reduce memory usage in Beagle tremendously. I've blogged about both tools in the past (mostly heap-buddy). You can search for them on http://joeshaw.org. Lluis wrote heap-shot and has blogged about it as well. Good luck, Joe ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hi, I finally managed to get heap-buddy working on my mac, it took a while, but i got there ;) After testing monotorrent for a while with the debugger enabled, nothing happened. Then i realised i wasn't using mono 1.2.3. A few minutes later i fired up monotorrent using mono 1.2.3 and within seconds had runaway memory usage. Here's the stats, maybe they'll make sense to someone, or fire alarm bells for someone else: I attached the seperate HeapBuddy reports to the email. Alan. On 3/29/07, Joe Shaw [EMAIL PROTECTED] wrote: Hi Alan, On 3/29/07, Alan McGovern [EMAIL PROTECTED] wrote: So, the thing is i can't track down what's causing the memory usage in monotorrent. I have no idea how to even start. I've tried using the built in profilers, but they require a graceful exit before they output their data, and unfortunately i can't get them a graceful exit ;) I would use the two external memory profilers for this: heap-buddy and heap-shot. Both are available from Mono SVN and straightforward to build and use. heap-buddy is a summarizing profiler. It collects allocation statistics while the program is running and writes out the aggregate info to a file which you analyze with a command-line tool after it's finished running. It is robust against your app exiting ungracefully. :) The tool has 4 basic modes: summary, history, types and backtraces. Summary gives you high-level info: how many objects/bytes were allocated, the final size of the heap, the number of GCs, etc. History gives you a timeline of each heap resize and GC, telling you the sizes of the heap, the number of objects and their size, and how it changed over time. You can see, for instance, that if your heap resizes 5 times in a row without a GC all within 2 seconds that you have a run away allocation pattern. :) Types will show you all the value types that have been created, how many total instances, the total bytes, the average size of one instance, and the average age (ie, the number of GCs) it lived for. If you're dealing with runaway memory, this will tell you what it is. The average age is good for finding memory leaks. Backtraces is like types, except it's more granular. It shows you the same info as types, but it breaks the types up by the call trace that allocated them. This is good when you're trying to find out what is allocating all those damn strings. :) The other tool is heap-shot, which is a snapshotting profiler. When you use it, you send SIGPROF to your running process which causes a file to be written out with all the information about allocations at that point. Afterward you can use the GUI to see where all the memory has gone. The basic display mode shows you all the types, along with the number of instances and the number of bytes these use. This is pretty similar to heap-buddy's types mode, except that it shows you the info for a snapshot in time, rather than the aggregate over the whole run of the program. The beauty of this tool is in the Reverse references mode, which tells you what objects hold references to a given type. So, if you reverse the references for string, you might see that 100 instances of string are being referenced by 50 instances of System.IO.FileInfo, and are using 2k of memory. As you drill down, you might see reference leaks (like those damned static ArrayLists and Hashtables) that continually grow without bound. These two tools (especially heap-shot) have helped us reduce memory usage in Beagle tremendously. I've blogged about both tools in the past (mostly heap-buddy). You can search for them on http://joeshaw.org. Lluis wrote heap-shot and has blogged about it as well. Good luck, Joe BackTraces Description: Binary data ResizeHistory Description: Binary data Summary Description: Binary data Types Description: Binary data ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
I ran it again, and came up with this (startling ;) ) figures for allocations: 1) type=System.Runtime.Remoti...ing.MonoMethodMessage 4185 1024.1M 256603.6 0.5 System.Threading.ThreadPool:QueueUserWorkItem System.Timers.Timer:StartTimer What the hell? 1 gig of allocations from a System.Timers.Timer? Whats going on here!? type=System.MonoType 2826 542.8M 201403.8 0.6 Half a gigabyte of System.MonoType's? What causes these to be allocated, and what can i do to stop this? Alan. On 3/30/07, Alan McGovern [EMAIL PROTECTED] wrote: Hi, I finally managed to get heap-buddy working on my mac, it took a while, but i got there ;) After testing monotorrent for a while with the debugger enabled, nothing happened. Then i realised i wasn't using mono 1.2.3. A few minutes later i fired up monotorrent using mono 1.2.3 and within seconds had runaway memory usage. Here's the stats, maybe they'll make sense to someone, or fire alarm bells for someone else: I attached the seperate HeapBuddy reports to the email. Alan. On 3/29/07, Joe Shaw [EMAIL PROTECTED] wrote: Hi Alan, On 3/29/07, Alan McGovern [EMAIL PROTECTED] wrote: So, the thing is i can't track down what's causing the memory usage in monotorrent. I have no idea how to even start. I've tried using the built in profilers, but they require a graceful exit before they output their data, and unfortunately i can't get them a graceful exit ;) I would use the two external memory profilers for this: heap-buddy and heap-shot. Both are available from Mono SVN and straightforward to build and use. heap-buddy is a summarizing profiler. It collects allocation statistics while the program is running and writes out the aggregate info to a file which you analyze with a command-line tool after it's finished running. It is robust against your app exiting ungracefully. :) The tool has 4 basic modes: summary, history, types and backtraces. Summary gives you high-level info: how many objects/bytes were allocated, the final size of the heap, the number of GCs, etc. History gives you a timeline of each heap resize and GC, telling you the sizes of the heap, the number of objects and their size, and how it changed over time. You can see, for instance, that if your heap resizes 5 times in a row without a GC all within 2 seconds that you have a run away allocation pattern. :) Types will show you all the value types that have been created, how many total instances, the total bytes, the average size of one instance, and the average age (ie, the number of GCs) it lived for. If you're dealing with runaway memory, this will tell you what it is. The average age is good for finding memory leaks. Backtraces is like types, except it's more granular. It shows you the same info as types, but it breaks the types up by the call trace that allocated them. This is good when you're trying to find out what is allocating all those damn strings. :) The other tool is heap-shot, which is a snapshotting profiler. When you use it, you send SIGPROF to your running process which causes a file to be written out with all the information about allocations at that point. Afterward you can use the GUI to see where all the memory has gone. The basic display mode shows you all the types, along with the number of instances and the number of bytes these use. This is pretty similar to heap-buddy's types mode, except that it shows you the info for a snapshot in time, rather than the aggregate over the whole run of the program. The beauty of this tool is in the Reverse references mode, which tells you what objects hold references to a given type. So, if you reverse the references for string, you might see that 100 instances of string are being referenced by 50 instances of System.IO.FileInfo, and are using 2k of memory. As you drill down, you might see reference leaks (like those damned static ArrayLists and Hashtables) that continually grow without bound. These two tools (especially heap-shot) have helped us reduce memory usage in Beagle tremendously. I've blogged about both tools in the past (mostly heap-buddy). You can search for them on http://joeshaw.org. Lluis wrote heap-shot and has blogged about it as well. Good luck, Joe ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hello, I would use the two external memory profilers for this: heap-buddy and heap-shot. Both are available from Mono SVN and straightforward to build and use. This was a great explanation of what Heap Buddy was, I have added a page to the Wiki with most of your email: http://www.mono-project.com/HeapBuddy One question: how do I enable the four different profiling modes for HeapBuddy? Miguel ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
Re: [Mono-dev] Need help tracking this bug...
Hello, I finally managed to get heap-buddy working on my mac, it took a while, but i got there ;) After testing monotorrent for a while with the debugger enabled, nothing happened. Then i realised i wasn't using mono 1.2.3. A few minutes later i fired up monotorrent using mono 1.2.3 and within seconds had runaway memory usage. Here's the stats, maybe they'll make sense to someone, or fire alarm bells for someone else: Could you run with heap-shot, and take snapshots every few seconds? With heap-shot you can see which objects are alive and do a diff between them and find out who keeps the references to the objects. It helps if you can also get the heap-shot-gui. ___ Mono-devel-list mailing list Mono-devel-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-devel-list
