Hi Matthieu, Please respond to the whole list so that everyone can share in the conversation.
When you start to get into parallel with the structured grids, I suggest you play around a bit with running pvserver in parallel, connect to it with the GUI, and play with some of the sources like Wavelet and Mandelbrot. As for comparisons with VisIt's in-situ visualization. We are working on getting interaction through the GUI to process the data directly coming out of the simulation without hitting the disk but it's not ready yet. There would be 2 use cases for that. The first is where it stops the simulation and waits for the user to finish their viz/analysis, i.e. like VisIt. The second would be where the user changes the outputting pipeline while the simulation code is running. The simulation code keeps running while the user is setting up new pipelines and the changes don't take effect until the next opportunity after the user adds their new pipeline or changes parameters to their existing pipeline. Our design started working with the use case that the user would submit their simulation+coprocessing as a batch job. This makes sense if the simulation code is "fast". For example, one of the simulation codes that we've been working with, PHASTA led by Ken Jansen at UC Boulder, is able to do an implicit time step solve (I believe it's implicit but don't quote me on that) in less than a second for a problem with hundreds of millions of unknowns. We've done this with up to 160,000 cores. While ParaView can still process that efficiently for many filters, it may take the user minutes to actually examine the data. That's obviously going to be a major waste of computational power if the supercomputer is waiting on the user before computing the next time step. In this case, the predefined pipeline works very well in that it completes the requested coprocessing and immediately gives control back to the simulation code to continue its processing. Hopefully this makes sense. I've inlined some comments below to make sure that I address your questions. Andy On Tue, Feb 7, 2012 at 8:52 AM, Matthieu Dorier <[email protected]>wrote: > Hi Andy, > > Thank you a lot. > > I'm not quiet sure to fully understand the SetWholeExtent stuff for > parallel runs but I will certainly come back to you in the future when I > effectively run in parallel. > > For now I have another question: I always assumed that in-situ > visualization with ParaView was done in a way similar than VisIt, with a > client that attempts to connect to perform interactive visualization. But > reading again all the Paraview tutorial, it seems I was wrong. > > Can you please tell me if my understanding is right: ParaView only feeds a > predefined pipeline (written in python or C++) with VTK objects, the > "normal" end of a pipeline consists in writing output (images, for > instance). > Yes, this is the current situation where you should consider the pipelines that are created by the script generator gui plugin to be static and to only output data files and screenshot images. You can modify the scripts to add in better control though. We also are working on allowing the user to change the pipeline but I don't know when this will be ready. > Using data staging, the pipeline's output is redirected to the > visualization cluster through sockets and the pvservers are used to > interact with the user. So there is a "static" part from the simulation to > staging where always the same coprocessing is performed, and an interactive > part from staging to user where the user can request more specific > visualization. Am I right? > This is also in the works where the data set or some subset of it will be staged from the compute cluster to a viz cluster for the user to interact with the "current" result. This will get done over the network and once the data has been passed to the viz cluster, the compute cluster can continue on with simulation computations, if that's what the user desires. > > Thanks, > > Matthieu > > > 2012/2/6 Andy Bauer <[email protected]> > >> Hi Matthieu, >> >> Your temperature array needs to be associated with your grid as point >> data. The temperature array will be stored as a derived class of >> vtkDataArray (I'll use a vtkDoubleArray). If you're storing your >> temperature data as a contiguous array with the same size as the number of >> points in your grid and you iterate through them such that you go through >> the x-direction fastest, then the y-direction next fastest, and finally >> z-direction, you'll probably want to use the SetArray() method to reuse the >> memory you're already using for your native storage. Then your >> wrapTemperature method would look like: >> >> vtkDataArray* wrapTemperature(double* temperature, vtkIdType >> temperatureSize) >> { >> vtkDoubleArray* temperatureArray = vtkDoubleArray::New(); >> temperatureArray->SetName("Temperature"); >> temperatureArray->SetArray(temperature, temperatureSize, 1); // use 1 >> to tell VTK not to delete temperature when it doesn't need the array anymore >> return temperatureArray; >> } >> >> Alternatively, creating a new array would look like: >> vtkDataArray* wrapTemperature(double* temperature, <more data>) >> { >> vtkDoubleArray* temperatureArray = vtkDoubleArray::New(); >> temperatureArray->SetNumberOfComponents(1); // assume temperature is a >> scalar tensor >> temperatureArray->SetNumberOfTuples(grid->GetNumberOfPoints()); >> temperatureArray->SetName("Temperature"); >> for(int zid=0;zid<PTZ;zid++) >> for(int yid=0;yid<PTY;yid++) >> for(int xid=0;xid<PTX;xid++) >> temperatureArray->SetTupleValue(xid+yid*PTX+zid*PTX*PTY, >> temperature+<corresponding index in array>); >> >> return temperatureArray; >> } >> >> Then your wrapMeshData() would look like: >> vtkDataObject* wrapMeshData() >> { >> vtkFloatArray* xCoords, yCoords, zCoords; >> xCoords = vtkFloatArray::New(); >> xCoords->setArray(mesh_x,PTX,1); >> yCoords = vtkFloatArray::New(); >> yCoords->setArray(mesh_y,PTY,1); >> zCoords = vtkFloatArray::New(); >> zCoords->setArray(mesh_z,PTZ,1); >> vtkRectilinearGrid *grid = vtkRectilinearGrid::New(); >> grid->setDimensions(PTX,PTY,PTZ); >> grid->setXCoordinates(xCoords); >> grid->setYCoordinates(yCoords); >> grid->setZCoordinates(zCoords); >> vtkDataArray* array = wrapTemperature(); >> grid->GetPointData()->AddArray(array); >> array->Delete(); // decrement array reference counter >> return (vtkDataObject*)grid; >> } >> >> Finally, don't forget to call "vtkCPDataDescription::SetWholeExtent(0, >> PTX-1, 0, PTY-1, 0, PTZ-1);". When running in parallel the input grid >> should be partitioned and wrapMeshData() would also call SetExtent() with >> that process's sub extent of the whole extent. >> >> Andy >> >> >> On Mon, Feb 6, 2012 at 5:49 AM, Matthieu Dorier < >> [email protected]> wrote: >> >>> Hello, >>> >>> I'm following the SC'10 tutorial with the help of the VTK doxygen, but I >>> don't understand how to do the following: >>> >>> I have a 3D rectilinear grid which coordinates are defined by mesh_x, >>> mesh_y and mesh_z (float arrays of dimensions PX, PY and PZ). I have a >>> variable "temperature" defined as a 3D array representing the temperature >>> at each point of the grid. Currently I have followed the slides 24 and 25 >>> from the SC'10 tutorial, at some point vtkDataObjects have to be created, >>> here is how I created the grid object : >>> >>> // This function is called to retrieve the mesh >>> vtkObject* wrapMeshData() >>> { >>> vtkFloatArray* xCoords, yCoords, zCoords; >>> xCoords = vtkFloatArray::New(); >>> xCoords->setArray(mesh_x,PTX,1); >>> yCoords = vtkFloatArray::New(); >>> yCoords->setArray(mesh_y,PTY,1); >>> zCoords = vtkFloatArray::New(); >>> zCoords->setArray(mesh_z,PTZ,1); >>> vtkRectilinearGrid *grid = vtkRectilinearGrid::New(); >>> grid->setDimensions(PTX,PTY,PTZ); >>> grid->setXCoordinates(xCoords); >>> grid->setYCoordinates(yCoords); >>> grid->setZCoordinates(zCoords); >>> return (vtkObject*)grid; >>> } >>> >>> Now I want to make a function that retrieves the temperature so that I >>> can map each data value to each point of the mesh, something like : >>> >>> vtkDataObject* wrapTemperature() { >>> // ??? >>> } >>> >>> How can I do that without copying the original array that I want to wrap? >>> Also what functions do I then call from the vtkCPDataDescription object? >>> (on the tutorial they call SetGrid, I guess there is something else to call >>> for the temperature field) >>> >>> Thank you for your help, >>> >>> Matthieu >>> >>> 2012/1/19 Berk Geveci <[email protected]> >>> >>>> Please feel free to ask questions if anything is not clear or if you >>>> need some help. We will be working on better documentation in the coming >>>> few months. >>>> >>>> Best, >>>> -berk >>>> >>>> >>>> On Wed, Jan 18, 2012 at 12:08 PM, Matthieu Dorier < >>>> [email protected]> wrote: >>>> >>>>> Thank you, this tutorial will help a lot. >>>>> >>>>> Matthieu >>>>> >>>>> >>>>> 2012/1/18 Andy Bauer <[email protected]> >>>>> >>>>>> The current main wiki page for coprocessing with ParaView is at >>>>>> http://paraview.org/Wiki/CoProcessing. In there it has links to 2 >>>>>> examples, a C++ driven example and a python driven example. You were >>>>>> looking at the C++ example. There's also a powerpoint presentation and >>>>>> some more code examples at >>>>>> http://www.paraview.org/Wiki/SC10_Coprocessing_Tutorial. We will >>>>>> but putting together a more extensive book but that probably won't be >>>>>> ready >>>>>> for a month or two. >>>>>> >>>>>> My suggestion would be to first work on creating a vtkDataObject to >>>>>> represent your data. Look at the powerpoint presentation for information >>>>>> on doing that and then maybe the VTK doxygen ( >>>>>> http://www.vtk.org/doc/nightly/html/classes.html) for more specific >>>>>> API questions. >>>>>> >>>>>> Andy >>>>>> >>>>>> On Wed, Jan 18, 2012 at 5:12 AM, Matthieu Dorier < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> I have difficulties to understand how to instrument a simulation >>>>>>> with ParaView in order to perform in-situ visualization. >>>>>>> Is there any documentation other than the simple example provided at >>>>>>> http://paraview.org/Wiki/Coprocessing_example ? >>>>>>> Thank you, >>>>>>> >>>>>>> -- >>>>>>> Matthieu Dorier >>>>>>> ENS Cachan, Brittany (Computer Science dpt.) >>>>>>> IRISA Rennes, Office C113 >>>>>>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307 >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Powered by www.kitware.com >>>>>>> >>>>>>> Visit other Kitware open-source projects at >>>>>>> http://www.kitware.com/opensource/opensource.html >>>>>>> >>>>>>> Please keep messages on-topic and check the ParaView Wiki at: >>>>>>> http://paraview.org/Wiki/ParaView >>>>>>> >>>>>>> Follow this link to subscribe/unsubscribe: >>>>>>> http://www.paraview.org/mailman/listinfo/paraview >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Matthieu Dorier >>>>> ENS Cachan, Brittany (Computer Science dpt.) >>>>> IRISA Rennes, Office C113 >>>>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307 >>>>> >>>>> _______________________________________________ >>>>> Powered by www.kitware.com >>>>> >>>>> Visit other Kitware open-source projects at >>>>> http://www.kitware.com/opensource/opensource.html >>>>> >>>>> Please keep messages on-topic and check the ParaView Wiki at: >>>>> http://paraview.org/Wiki/ParaView >>>>> >>>>> Follow this link to subscribe/unsubscribe: >>>>> http://www.paraview.org/mailman/listinfo/paraview >>>>> >>>>> >>>> >>> >>> >>> -- >>> Matthieu Dorier >>> ENS Cachan, Brittany (Computer Science dpt.) >>> IRISA Rennes, Office C113 >>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307 >>> >> >> > > > -- > Matthieu Dorier > ENS Cachan, Brittany (Computer Science dpt.) > IRISA Rennes, Office C113 > http://perso.eleves.bretagne.ens-cachan.fr/~mdori307 >
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
