Use swap space. Matt.
From: <hcp-users-boun...@humanconnectome.org<mailto:hcp-users-boun...@humanconnectome.org>> on behalf of Timothy Coalson <tsc...@mst.edu<mailto:tsc...@mst.edu>> Date: Wednesday, October 24, 2018 at 4:57 PM To: Reza Rajimehr <rajim...@gmail.com<mailto:rajim...@gmail.com>> Cc: "hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>" <hcp-users@humanconnectome.org<mailto:hcp-users@humanconnectome.org>> Subject: Re: [HCP-Users] Question about dense functional connectome The cifti format doesn't support that, because it is designed to allow use without loading the entire file into memory - there is no obvious file organization that would allow efficient loading of a full row from a single triangular file, when seek times are nontrivial (rotating disks). I don't believe we have these files in any other format. I'm also not sure how easy it would be to work on such a matrix in matlab. If wb_command is capable of doing the operations you want, it is usually possible for it to do them without loading the entire cifti file into memory at once. wb_view will also display maps from it without loading the entire file into memory. DDR4 has 16GB modules available at reasonable prices, which should allow recent computers with even 4 slots for memory to expand to 64GB. You should also consider the "high-end desktop" platforms (threadripper, lga2066, lga2011), which typically offer 8 memory slots, and processors with more cores. Tim On Wed, Oct 24, 2018 at 3:56 PM, Reza Rajimehr <rajim...@gmail.com<mailto:rajim...@gmail.com>> wrote: Hi, We are trying to unzip and load dense functional connectome (HCP_S900_820_rfMRI_MSMAll_groupPCA_d4500ROW_zcorr.zip or HCP_S900_820_rfMRI_MSMSulc_groupPCA_d4500ROW_zcorr.zip from https://www.humanconnectome.org/study/hcp-young-adult/article/announcing-release-of-s900-ptn-and-other-group-average-data) in Matlab on a system that has 32 GB RAM. This is of course impossible due to the fact that the files are ~33 GB in size. Due to matrix symmetry, it would be sufficient to store only the upper or lower triangular part of a functional connectivity matrix, reducing memory occupancy by about 50% (Loewe et al., 2016). Does HCP provide such files that could be memory-friendly? Best, Reza _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org<mailto:HCP-Users@humanconnectome.org> http://lists.humanconnectome.org/mailman/listinfo/hcp-users _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org<mailto:HCP-Users@humanconnectome.org> http://lists.humanconnectome.org/mailman/listinfo/hcp-users ________________________________ The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail. _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org http://lists.humanconnectome.org/mailman/listinfo/hcp-users
