Cameron Huddlestone-Holmes: |Hello DX users, | |I am working with a large dataset generated by high-resolution CT scans. It |consists of 918 slices @ 1024x1024, 16-bit (~1.7Gb). Obviously this causes |huge problems with memory, so I was hoping to make subsets of the data to |regions of interest using the include module.
You might want to consider using Slab instead of Include. Include is a very capable module, but if you're not careful (e.g. you use the defaults) it'll convert your data to unstructured and run you out of memory real fast. OTOH, if you are careful, it's not realy going to save you any memory. Also consider Reduce. |regions of interest using the include module. We are currently tweaking our |SGI Origin 3400 to increase the amount of RAM available to dx. | |What I was wondering is there any way, other than trial and error, to |estimate the maximum amount of memory required to run a visual program, or |by an individual module. Once you understand what DX is doing to your data, I find it's usually pretty intuitive. You're running on SGI, so another thought might be to build a 64-bit DX (configure --with-large-arenas in CVS) which will get you > ~1.5GB total data (e.g. 7GB), but won't remove the 2GB limitation on individual objects. I haven't gotten back to chasing down the UI bugs, but the executive seemed pretty sound. You might be able to connect a 32-bit dxui front-end to a 64-bit dxexec back-end using the standard Connection->Start Server... but I haven't tried it. Let me know if you want more info. |The modules I have in my programs (that I assume use significant memory) are |- |Import |Extract |Statistics |Mark |Unmark |Export Negligible. |Compute The size of the data array you're creating |Include When given a regular grid, maybe a little (cull=0), or a horrendous amount (cull=1), depending on the data. Other considerations. Under the Edit menu, select all modules and optimize cacheability. You can also turn off caching if memory gets really tight. Consider using a 64-bit executive, or Reducing/Slabing/Slice'ing your data before passing it through your network. Randy -- Randall Hopper (mailto:[EMAIL PROTECTED]) Lockheed Martin Operation Support EPA Scientific Visualization Center US EPA N127-01; RTP, NC 27711
