Is there a way to import/map memory from a process (data acquisition) such that an MPI program could 'take' or see that memory?
We have a need to do data acquisition at the rate of .7TB/s and need todo some shuffles/computation on these data, some of the nodes are directly connected to the device, and some will do processing. Here is the proposed flow: * Data collector nodes runs process collecting data from device * Those nodes somehow pass the data to an MPI job running on these nodes and a number of other nodes (cpu need for filterting is greater than what the 16 data nodes can provide). One thought is to have the data collector processes be threads inside the MPI job running across all nodes, but was curious is there is a way to pass data still in memory (to much to hit disk) to the running MPI filter job. Thanks! Brock Palen www.umich.edu/~brockp CAEN Advanced Computing XSEDE Campus Champion bro...@umich.edu (734)936-1985
signature.asc
Description: Message signed with OpenPGP using GPGMail
