It should compute it one row at a time, so as long as you have enough rows that it divides the total cifti file into a managable row size, you should be fine. You can always try it on your local machine, and see if it starts allocating swap (cifti-math, along with the other math commands, aren't parallel anyway, it should usually be IO bound since the amount of processing to do is exactly proportional to the size of the data files, so unless you are running multiple commands in parallel, it probably doesn't need to be on a cluster). In general, I try to write cifti commands so that they compute/write cifti files one row at a time during computation, so they don't use as much memory. Correlation is a notable exception, since the output is symmetric, retaining some/all of the results in memory allows it to finish faster.
Note, however, that metric/volume inputs/outputs always use the memory of the uncompressed data (we don't do on-disk reading/writing for them), and most cifti commands that do spatial things like smoothing or gradient separate the cifti into in-memory metric/volume files one at a time, which may take a fair amount of memory. I hadn't thought of making a "count nonzero" reduce operation, but it is rather easy to add more reduce operations. Tim On Sat, Oct 5, 2013 at 11:07 AM, Colin Reveley <[email protected]> wrote: > Hi - > > I'm wondering (before I try and damage remote equipment) > > when running wb_command -cifti-math > > is it necessary that the cifti(s) fit into RAM memory? > > what I want to do is take a very large dconn.nii file, and binarize it > such that all non-zero entries are 1 and all 0 entries stay 0. > > this is, obviously, an intermediate step to something else (tracking stats > from vertex seeds). > > In an ideal world I'd also like to divide the value in each cell by > another cifti of the same dimensions. > > if -cifti-math is able to avoid RAM (like cifti-reduce) then great. If not > I need another way or I risk disrupting the university cluster. > > the basic task is to replace all non-zero entries with 1 without requiring > more than say 1/10 of the file size in RAM. > > if that is not possible then, if we call the binarised cifti above "A" > then the output of -cifti-reduce A.dconn.nii SUM > > obtained by any means would be great and a big help > > obviously there is more detail to the problem/questions that might help. > But I'm trying not to explicitly be "unsupported" at this point in case the > question can be answered in a "supported" way. > > best, > > Colin > > > _______________________________________________ > caret-users mailing list > [email protected] > http://brainvis.wustl.edu/mailman/listinfo/caret-users > >
_______________________________________________ caret-users mailing list [email protected] http://brainvis.wustl.edu/mailman/listinfo/caret-users
