>> Since time is of the essence, I created a Singleton .NET Remote server Oops sorry. I meant SingleCall, Outlook autocorrected me (grrrrr), more info here: http://msdn.microsoft.com/en-us/library/ms973864.aspx
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Alex Rufon Sent: Wednesday, September 08, 2010 8:41 PM To: [email protected] Subject: [Jgeneral] J, Memory usage and .NET Remoting As I have described before, I've been moving the execution of J scripts from the Application Server down to the client machines. This has worked well for me considering that J would only execute for less than a minute to around 3 minutes maximum. This meant that the data I'm feeding J is relatively small ... J would normally not go beyond 64MB memory consumption (as shown by the Task Manager). If your curious, I run J on Win2K to Win7 machines with a minimum of 256MB RAM. J is not the primary application but is called by a Smart Client application built with C# using .NET Interopt through COM/OLEAUTOMATION. So last week (or the week before), one of my boss asked me to re-compute raw material consumption (for a set of apparel/shirts) on 3 years' worth of data. So I modified some existing code and computed 4,700+ styles in one J session. As soon as the Task Manager reported that J is allocating more than 50MB of memory, computation started to slow down. I am running this on my development machine which is a Lenovo T61 with 3GB RAM and 2.20GHz Core 2 Duo CPU. The PC itself is still useable and is responsive (I can still browse the internet, check & read emails, etc.) but J itself is not responding anymore and the usual 30 second computation per Style is worsening to 3minutes (remember I have to process almost 5K styles). I do know why its slowing down (I have a vector of unique strings that keeps getting bigger as more new strings are added) but this is a one off deal that fixing it would be too much useless work. My first reaction was just to move J to one of our production app server. Initially, it looked good but I since I was using a single instance of J ... thing went bad as soon as the memory allocation went over 100mb. Since time is of the essence, I created a Singleton .NET Remote server. The key concept here is that every time a client calls the remote server to compute, a new instance of J is created and as soon as it is done, that instance is discarded and awaits garbage collection. The performance difference was very significant. More so when I split the styles in half and got 2 client PC's to push the data to the server which were running on 2x Intel Xeon Quadcore CPU's. So what I'm saying is that even though I got a big iron application server, running all my process on a single instance of J was the wrong way to go. I believe that a clone server with the Singleton .NET Remote server configuration would beat the HP Proliant Server even with the overhead of creating new instance of J (and populating it with base data) every time. I hope a good parallel execution framework for J come out soon. ;) r/Alex ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
