Hi Andy, Thank you for your answer (and sorry, I didn't notice I did a "reply" instead of a "reply to all").
Matthieu 2012/2/7 Andy Bauer <[email protected]> > Hi Matthieu, > > Please respond to the whole list so that everyone can share in the > conversation. > > When you start to get into parallel with the structured grids, I suggest > you play around a bit with running pvserver in parallel, connect to it with > the GUI, and play with some of the sources like Wavelet and Mandelbrot. > > As for comparisons with VisIt's in-situ visualization. We are working on > getting interaction through the GUI to process the data directly coming out > of the simulation without hitting the disk but it's not ready yet. There > would be 2 use cases for that. The first is where it stops the simulation > and waits for the user to finish their viz/analysis, i.e. like VisIt. The > second would be where the user changes the outputting pipeline while the > simulation code is running. The simulation code keeps running while the > user is setting up new pipelines and the changes don't take effect until > the next opportunity after the user adds their new pipeline or changes > parameters to their existing pipeline. Our design started working with the > use case that the user would submit their simulation+coprocessing as a > batch job. This makes sense if the simulation code is "fast". For > example, one of the simulation codes that we've been working with, PHASTA > led by Ken Jansen at UC Boulder, is able to do an implicit time step solve > (I believe it's implicit but don't quote me on that) in less than a second > for a problem with hundreds of millions of unknowns. We've done this with > up to 160,000 cores. While ParaView can still process that efficiently for > many filters, it may take the user minutes to actually examine the data. > That's obviously going to be a major waste of computational power if the > supercomputer is waiting on the user before computing the next time step. > In this case, the predefined pipeline works very well in that it completes > the requested coprocessing and immediately gives control back to the > simulation code to continue its processing. Hopefully this makes sense. > > I've inlined some comments below to make sure that I address your > questions. > > Andy > > On Tue, Feb 7, 2012 at 8:52 AM, Matthieu Dorier <[email protected] > > wrote: > >> Hi Andy, >> >> Thank you a lot. >> >> I'm not quiet sure to fully understand the SetWholeExtent stuff for >> parallel runs but I will certainly come back to you in the future when I >> effectively run in parallel. >> >> For now I have another question: I always assumed that in-situ >> visualization with ParaView was done in a way similar than VisIt, with a >> client that attempts to connect to perform interactive visualization. But >> reading again all the Paraview tutorial, it seems I was wrong. >> >> Can you please tell me if my understanding is right: ParaView only feeds >> a predefined pipeline (written in python or C++) with VTK objects, the >> "normal" end of a pipeline consists in writing output (images, for >> instance). >> > Yes, this is the current situation where you should consider the pipelines > that are created by the script generator gui plugin to be static and to > only output data files and screenshot images. You can modify the scripts > to add in better control though. We also are working on allowing the user > to change the pipeline but I don't know when this will be ready. > > >> Using data staging, the pipeline's output is redirected to the >> visualization cluster through sockets and the pvservers are used to >> interact with the user. So there is a "static" part from the simulation to >> staging where always the same coprocessing is performed, and an interactive >> part from staging to user where the user can request more specific >> visualization. Am I right? >> > This is also in the works where the data set or some subset of it will be > staged from the compute cluster to a viz cluster for the user to interact > with the "current" result. This will get done over the network and once > the data has been passed to the viz cluster, the compute cluster can > continue on with simulation computations, if that's what the user desires. > >> >> Thanks, >> >> Matthieu >> >> >> 2012/2/6 Andy Bauer <[email protected]> >> >>> Hi Matthieu, >>> >>> Your temperature array needs to be associated with your grid as point >>> data. The temperature array will be stored as a derived class of >>> vtkDataArray (I'll use a vtkDoubleArray). If you're storing your >>> temperature data as a contiguous array with the same size as the number of >>> points in your grid and you iterate through them such that you go through >>> the x-direction fastest, then the y-direction next fastest, and finally >>> z-direction, you'll probably want to use the SetArray() method to reuse the >>> memory you're already using for your native storage. Then your >>> wrapTemperature method would look like: >>> >>> vtkDataArray* wrapTemperature(double* temperature, vtkIdType >>> temperatureSize) >>> { >>> vtkDoubleArray* temperatureArray = vtkDoubleArray::New(); >>> temperatureArray->SetName("Temperature"); >>> temperatureArray->SetArray(temperature, temperatureSize, 1); // use 1 >>> to tell VTK not to delete temperature when it doesn't need the array anymore >>> return temperatureArray; >>> } >>> >>> Alternatively, creating a new array would look like: >>> vtkDataArray* wrapTemperature(double* temperature, <more data>) >>> { >>> vtkDoubleArray* temperatureArray = vtkDoubleArray::New(); >>> temperatureArray->SetNumberOfComponents(1); // assume temperature is >>> a scalar tensor >>> temperatureArray->SetNumberOfTuples(grid->GetNumberOfPoints()); >>> temperatureArray->SetName("Temperature"); >>> for(int zid=0;zid<PTZ;zid++) >>> for(int yid=0;yid<PTY;yid++) >>> for(int xid=0;xid<PTX;xid++) >>> temperatureArray->SetTupleValue(xid+yid*PTX+zid*PTX*PTY, >>> temperature+<corresponding index in array>); >>> >>> return temperatureArray; >>> } >>> >>> Then your wrapMeshData() would look like: >>> vtkDataObject* wrapMeshData() >>> { >>> vtkFloatArray* xCoords, yCoords, zCoords; >>> xCoords = vtkFloatArray::New(); >>> xCoords->setArray(mesh_x,PTX,1); >>> yCoords = vtkFloatArray::New(); >>> yCoords->setArray(mesh_y,PTY,1); >>> zCoords = vtkFloatArray::New(); >>> zCoords->setArray(mesh_z,PTZ,1); >>> vtkRectilinearGrid *grid = vtkRectilinearGrid::New(); >>> grid->setDimensions(PTX,PTY,PTZ); >>> grid->setXCoordinates(xCoords); >>> grid->setYCoordinates(yCoords); >>> grid->setZCoordinates(zCoords); >>> vtkDataArray* array = wrapTemperature(); >>> grid->GetPointData()->AddArray(array); >>> array->Delete(); // decrement array reference counter >>> return (vtkDataObject*)grid; >>> } >>> >>> Finally, don't forget to call "vtkCPDataDescription::SetWholeExtent(0, >>> PTX-1, 0, PTY-1, 0, PTZ-1);". When running in parallel the input grid >>> should be partitioned and wrapMeshData() would also call SetExtent() with >>> that process's sub extent of the whole extent. >>> >>> Andy >>> >>> >>> On Mon, Feb 6, 2012 at 5:49 AM, Matthieu Dorier < >>> [email protected]> wrote: >>> >>>> Hello, >>>> >>>> I'm following the SC'10 tutorial with the help of the VTK doxygen, but >>>> I don't understand how to do the following: >>>> >>>> I have a 3D rectilinear grid which coordinates are defined by mesh_x, >>>> mesh_y and mesh_z (float arrays of dimensions PX, PY and PZ). I have a >>>> variable "temperature" defined as a 3D array representing the temperature >>>> at each point of the grid. Currently I have followed the slides 24 and 25 >>>> from the SC'10 tutorial, at some point vtkDataObjects have to be created, >>>> here is how I created the grid object : >>>> >>>> // This function is called to retrieve the mesh >>>> vtkObject* wrapMeshData() >>>> { >>>> vtkFloatArray* xCoords, yCoords, zCoords; >>>> xCoords = vtkFloatArray::New(); >>>> xCoords->setArray(mesh_x,PTX,1); >>>> yCoords = vtkFloatArray::New(); >>>> yCoords->setArray(mesh_y,PTY,1); >>>> zCoords = vtkFloatArray::New(); >>>> zCoords->setArray(mesh_z,PTZ,1); >>>> vtkRectilinearGrid *grid = vtkRectilinearGrid::New(); >>>> grid->setDimensions(PTX,PTY,PTZ); >>>> grid->setXCoordinates(xCoords); >>>> grid->setYCoordinates(yCoords); >>>> grid->setZCoordinates(zCoords); >>>> return (vtkObject*)grid; >>>> } >>>> >>>> Now I want to make a function that retrieves the temperature so that I >>>> can map each data value to each point of the mesh, something like : >>>> >>>> vtkDataObject* wrapTemperature() { >>>> // ??? >>>> } >>>> >>>> How can I do that without copying the original array that I want to >>>> wrap? >>>> Also what functions do I then call from the vtkCPDataDescription >>>> object? (on the tutorial they call SetGrid, I guess there is something else >>>> to call for the temperature field) >>>> >>>> Thank you for your help, >>>> >>>> Matthieu >>>> >>>> 2012/1/19 Berk Geveci <[email protected]> >>>> >>>>> Please feel free to ask questions if anything is not clear or if you >>>>> need some help. We will be working on better documentation in the coming >>>>> few months. >>>>> >>>>> Best, >>>>> -berk >>>>> >>>>> >>>>> On Wed, Jan 18, 2012 at 12:08 PM, Matthieu Dorier < >>>>> [email protected]> wrote: >>>>> >>>>>> Thank you, this tutorial will help a lot. >>>>>> >>>>>> Matthieu >>>>>> >>>>>> >>>>>> 2012/1/18 Andy Bauer <[email protected]> >>>>>> >>>>>>> The current main wiki page for coprocessing with ParaView is at >>>>>>> http://paraview.org/Wiki/CoProcessing. In there it has links to 2 >>>>>>> examples, a C++ driven example and a python driven example. You were >>>>>>> looking at the C++ example. There's also a powerpoint presentation and >>>>>>> some more code examples at >>>>>>> http://www.paraview.org/Wiki/SC10_Coprocessing_Tutorial. We will >>>>>>> but putting together a more extensive book but that probably won't be >>>>>>> ready >>>>>>> for a month or two. >>>>>>> >>>>>>> My suggestion would be to first work on creating a vtkDataObject to >>>>>>> represent your data. Look at the powerpoint presentation for >>>>>>> information >>>>>>> on doing that and then maybe the VTK doxygen ( >>>>>>> http://www.vtk.org/doc/nightly/html/classes.html) for more specific >>>>>>> API questions. >>>>>>> >>>>>>> Andy >>>>>>> >>>>>>> On Wed, Jan 18, 2012 at 5:12 AM, Matthieu Dorier < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> I have difficulties to understand how to instrument a simulation >>>>>>>> with ParaView in order to perform in-situ visualization. >>>>>>>> Is there any documentation other than the simple example provided >>>>>>>> at http://paraview.org/Wiki/Coprocessing_example ? >>>>>>>> Thank you, >>>>>>>> >>>>>>>> -- >>>>>>>> Matthieu Dorier >>>>>>>> ENS Cachan, Brittany (Computer Science dpt.) >>>>>>>> IRISA Rennes, Office C113 >>>>>>>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307 >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Powered by www.kitware.com >>>>>>>> >>>>>>>> Visit other Kitware open-source projects at >>>>>>>> http://www.kitware.com/opensource/opensource.html >>>>>>>> >>>>>>>> Please keep messages on-topic and check the ParaView Wiki at: >>>>>>>> http://paraview.org/Wiki/ParaView >>>>>>>> >>>>>>>> Follow this link to subscribe/unsubscribe: >>>>>>>> http://www.paraview.org/mailman/listinfo/paraview >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Matthieu Dorier >>>>>> ENS Cachan, Brittany (Computer Science dpt.) >>>>>> IRISA Rennes, Office C113 >>>>>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307 >>>>>> >>>>>> _______________________________________________ >>>>>> Powered by www.kitware.com >>>>>> >>>>>> Visit other Kitware open-source projects at >>>>>> http://www.kitware.com/opensource/opensource.html >>>>>> >>>>>> Please keep messages on-topic and check the ParaView Wiki at: >>>>>> http://paraview.org/Wiki/ParaView >>>>>> >>>>>> Follow this link to subscribe/unsubscribe: >>>>>> http://www.paraview.org/mailman/listinfo/paraview >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Matthieu Dorier >>>> ENS Cachan, Brittany (Computer Science dpt.) >>>> IRISA Rennes, Office C113 >>>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307 >>>> >>> >>> >> >> >> -- >> Matthieu Dorier >> ENS Cachan, Brittany (Computer Science dpt.) >> IRISA Rennes, Office C113 >> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307 >> > > -- Matthieu Dorier ENS Cachan, Brittany (Computer Science dpt.) IRISA Rennes, Office C113 http://perso.eleves.bretagne.ens-cachan.fr/~mdori307
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
