Re: [OpenDX2-users] OpenDX development
Hi, Marco - I'll be distributing the code I've been working on - I just want to do a bit more testing. I think I'll be using a SourceForge repository. The changes I've been making affect the standard shared-memory model of OpenDX. The work I did a long time ago on an MPI-based distributed-memory version was built on top of the shared-memory model; whether that is multi-process or multi-thread should be immaterial. However, I'm not sure I'm going to try to resurrect that stuff; it was only a prototype and never really became real. I don't believe the system has been flushing the cache at all, in the past. Thats why it would run until it ran out of memory, take a few half-hearted efforts at scavenging, and then stop. At which point you could explicitly flush the cache, and it'd proceed along just fine - until it ran out of memory again. I have a test progeam that steps up and back through 1000 isovalues; the old code would not run very long before stopping with an out-of-memory error. With the fix in, even in very constrained memory it keeps running a *long* time. Greg Gregory D. Abram Texas Advanced Computing Center JJ Pickle Research Campus - ROC 1.101 Bldg 196 10100 Burnet Road Austin, Texas 78758-4497 g...@tacc.utexas.edu (512) 471-8196 From: Marco Morandini [morand...@aero.polimi.it] Sent: Monday, October 26, 2009 5:49 AM To: Greg Abram Cc: opendx2-users@lists.berlios.de Subject: Re: [OpenDX2-users] OpenDX development Greg Abram wrote: > Hi - > > Anyone out there? I'm wondering if anyone is interested in some > work I've been doing on OpenDX - I've replaced the multi-process > model with a multi-threaded one, enabling a much (much!) simplified > memory manager. This enables me to remove the limit on memory size > (though I bet you'd better not allocate any individual object larger > than 2GB). It'll just consume virtual-memory page-space until it > hits an arbitrary limit (suggestions?) when it will begin to flush > the cache. Fragmentation is also much less of a problem. > It would be great if you could post the patches, so that they remain archived somewhere, waiting for someone to re-vitalize the project. Do the multi-thread model rules out the possibility to use mpi? I'm also very interested in the cache bug you mentioned in the forum. Thanks, Marco ___ OpenDX2-users mailing list OpenDX2-users@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opendx2-users
Re: [OpenDX2-users] OpenDX development
Greg Abram wrote: > Hi - > > Anyone out there? I'm wondering if anyone is interested in some > work I've been doing on OpenDX - I've replaced the multi-process > model with a multi-threaded one, enabling a much (much!) simplified > memory manager. This enables me to remove the limit on memory size > (though I bet you'd better not allocate any individual object larger > than 2GB). It'll just consume virtual-memory page-space until it > hits an arbitrary limit (suggestions?) when it will begin to flush > the cache. Fragmentation is also much less of a problem. > It would be great if you could post the patches, so that they remain archived somewhere, waiting for someone to re-vitalize the project. Do the multi-thread model rules out the possibility to use mpi? I'm also very interested in the cache bug you mentioned in the forum. Thanks, Marco ___ OpenDX2-users mailing list OpenDX2-users@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opendx2-users
Re: [OpenDX2-users] OpenDX development
Hi Greg! > Anyone out there? I'm wondering if anyone is interested in some work I've > been doing on OpenDX - I've replaced the multi-process model with a > multi-threaded one, enabling a much (much!) simplified memory manager. This > enables me to remove the limit on memory size (though I bet you'd better not > allocate any individual object larger than 2GB). It'll just consume > virtual-memory page-space until it hits an arbitrary limit (suggestions?) > when it will begin to flush the cache. Fragmentation is also much less of a > problem. > Sounds great. Just one thought - is it still possible to use the multi-thread one, or did you really replace it? I am asking because I remember when I was working on clusters, that for some solution process migration works flawless, but process migration due to shared memory is not implemented on some cluster platforms (that was the case for mosix when I last used it). About the frequency of postings here, obviously most discussion moved to the web-based forums. I also prefer mailing lists, but I am aware that today web-forums are easier to find and use by many. Would be great to link the mailing list and the forums, of course... Cheers, Lars. ___ OpenDX2-users mailing list OpenDX2-users@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opendx2-users
[OpenDX2-users] OpenDX development
Hi - Anyone out there? I'm wondering if anyone is interested in some work I've been doing on OpenDX - I've replaced the multi-process model with a multi-threaded one, enabling a much (much!) simplified memory manager. This enables me to remove the limit on memory size (though I bet you'd better not allocate any individual object larger than 2GB). It'll just consume virtual-memory page-space until it hits an arbitrary limit (suggestions?) when it will begin to flush the cache. Fragmentation is also much less of a problem. Gregory D. Abram Texas Advanced Computing Center JJ Pickle Research Campus - ROC 1.101 Bldg 196 10100 Burnet Road Austin, Texas 78758-4497 g...@tacc.utexas.edu (512) 471-8196 ___ OpenDX2-users mailing list OpenDX2-users@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opendx2-users
