If your swap space isn't on an SSD, then it will be extremely slow to do anything on it.
Tim On Wed, Oct 24, 2018 at 7:19 PM, Glasser, Matthew <glass...@wustl.edu> wrote: > Use swap space. > > Matt. > > From: <hcp-users-boun...@humanconnectome.org> on behalf of Timothy > Coalson <tsc...@mst.edu> > Date: Wednesday, October 24, 2018 at 4:57 PM > To: Reza Rajimehr <rajim...@gmail.com> > Cc: "hcp-users@humanconnectome.org" <hcp-users@humanconnectome.org> > Subject: Re: [HCP-Users] Question about dense functional connectome > > The cifti format doesn't support that, because it is designed to allow use > without loading the entire file into memory - there is no obvious file > organization that would allow efficient loading of a full row from a single > triangular file, when seek times are nontrivial (rotating disks). I don't > believe we have these files in any other format. I'm also not sure how > easy it would be to work on such a matrix in matlab. If wb_command is > capable of doing the operations you want, it is usually possible for it to > do them without loading the entire cifti file into memory at once. wb_view > will also display maps from it without loading the entire file into memory. > > DDR4 has 16GB modules available at reasonable prices, which should allow > recent computers with even 4 slots for memory to expand to 64GB. You > should also consider the "high-end desktop" platforms (threadripper, > lga2066, lga2011), which typically offer 8 memory slots, and processors > with more cores. > > Tim > > > On Wed, Oct 24, 2018 at 3:56 PM, Reza Rajimehr <rajim...@gmail.com> wrote: > >> Hi, >> >> We are trying to unzip and load dense functional connectome >> (HCP_S900_820_rfMRI_MSMAll_groupPCA_d4500ROW_zcorr.zip or >> HCP_S900_820_rfMRI_MSMSulc_groupPCA_d4500ROW_zcorr.zip from >> https://www.humanconnectome.org/study/hcp-young-adult/articl >> e/announcing-release-of-s900-ptn-and-other-group-average-data) in Matlab >> on a system that has 32 GB RAM. This is of course impossible due to the >> fact that the files are ~33 GB in size. Due to matrix symmetry, it would be >> sufficient to store only the upper or lower triangular part of a functional >> connectivity matrix, reducing memory occupancy by about 50% (Loewe et al., >> 2016). Does HCP provide such files that could be memory-friendly? >> >> Best, >> Reza >> >> _______________________________________________ >> HCP-Users mailing list >> HCP-Users@humanconnectome.org >> http://lists.humanconnectome.org/mailman/listinfo/hcp-users >> > > _______________________________________________ > HCP-Users mailing list > HCP-Users@humanconnectome.org > http://lists.humanconnectome.org/mailman/listinfo/hcp-users > > > ------------------------------ > > The materials in this message are private and may contain Protected > Healthcare Information or other information of a sensitive nature. If you > are not the intended recipient, be advised that any unauthorized use, > disclosure, copying or the taking of any action in reliance on the contents > of this information is strictly prohibited. If you have received this email > in error, please immediately notify the sender via telephone or return mail. > _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org http://lists.humanconnectome.org/mailman/listinfo/hcp-users
