Re: [Paraview] Why is PKdTree used for IMAGE Volume Rendering?
Do you have any unstructured data being rendering in the scene? KdTree is used to distribute that unstructured data in accordance with the partitions provided by the structured data. Utkarsh On Thu, Feb 12, 2009 at 2:09 PM, Kevin H. Hobbs hob...@ohiou.edu wrote: I just tried to use ParaView from Tuesday's CVS (The network was down last night.) on both CPU cores to volume render a large (1/4 the size of available RAM) image. As soon as I selected volume render for the representation I saw the status bar show PKdTree and moments later all of the system memory was consumed and the system started swapping. I had to kill paraview and the pvserver processes from the console to end the swap-death. I reran paraview with only one CPU core and proceeded as normal. My question is: why is PKdTree needed at all for image data? For unstructured data it of course makes sense but for image data it just wastes a lot of RAM. I haven't tried parallel image volume rendering in a while so I don't know when this showed up. ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
Re: [Paraview] Why is PKdTree used for IMAGE Volume Rendering?
On Thu, 2009-02-12 at 14:26 -0500, Utkarsh Ayachit wrote: Do you have any unstructured data being rendering in the scene? No, just one big .vti image. signature.asc Description: This is a digitally signed message part ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
Re: [Paraview] Why is PKdTree used for IMAGE Volume Rendering?
Kevin, The KdTree is still built (using the partitions from the structured dataset set itself -- hence isn't a very compute intensive task) just in case it's needed to partition any unstructured data, hence you're seeing the PKdTree progress. But I don't think that's where it is hanging. Is the progress bar saying PKtree:100 ? It may actually have frozen in the volume rendering code. Is it possible to attach a debugger to any of the server processes and see where it's stuck (provide the stack trace if possible)? Utkarsh On Thu, Feb 12, 2009 at 2:37 PM, Kevin H. Hobbs hob...@ohiou.edu wrote: On Thu, 2009-02-12 at 14:26 -0500, Utkarsh Ayachit wrote: Do you have any unstructured data being rendering in the scene? No, just one big .vti image. ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
Re: [Paraview] Why is PKdTree used for IMAGE Volume Rendering?
Ah and there's the bad news. In case of image data for the if condition at vtkPKdTree.cxx:435, I'd expect the condition to succeed and go to this-ProcessUserDefinedCuts(..) however, it seems like that condition is failing and it's going to this-MultiProcessBuildLocator(..) which is used for unstructured datasets. Are you able to reproduce the behavior with a simpler dataset (say wavelet) or something that;s already in ParaViewData, or some dataset that I can try it with? Also give me all the steps to reproduce the problem (since I am not able to reproduce it with 5 procs using Wavelet source). Utkarsh On Thu, Feb 12, 2009 at 3:24 PM, Kevin H. Hobbs hob...@ohiou.edu wrote: On Thu, 2009-02-12 at 14:56 -0500, Utkarsh Ayachit wrote: Kevin, The KdTree is still built (using the partitions from the structured dataset set itself -- hence isn't a very compute intensive task) just in case it's needed to partition any unstructured data, hence you're seeing the PKdTree progress. But I don't think that's where it is hanging. Is the progress bar saying PKtree:100 ? No, it only made it to 58 It may actually have frozen in the volume rendering code. Is it possible to attach a debugger to any of the server processes and see where it's stuck (provide the stack trace if possible)? Paraview stops completely as soon as I attach but here is the first bt before it really hung up: (gdb) bt #0 vtkDataArrayTemplatedouble::DataChanged (this=0x2e89fa0) at /home/kevin/kitware/ParaView3/VTK/Common/vtkDataArrayTemplate.txx:1202 #1 0x7f035a9cc221 in vtkPoints::SetPoint () at /home/kevin/kitware/ParaView3/VTK/Common/vtkPoints.h:126 #2 vtkImageData::GetCell (this=0x2e897d0, cellId=48799904) at /home/kevin/kitware/ParaView3/VTK/Filtering/vtkImageData.cxx:392 #3 0x7f035aa00bed in vtkKdTree::ComputeCellCenters (this=0x28853d0, set=0x0) at /home/kevin/kitware/ParaView3/VTK/Filtering/vtkKdTree.cxx:677 #4 0x7f035eee71eb in vtkPKdTree::MultiProcessBuildLocator (this=0x28853d0, volBounds=0x2e91140) at /home/kevin/kitware/ParaView3/VTK/Parallel/vtkPKdTree.cxx:498 #5 0x7f035eee7683 in vtkPKdTree::BuildLocator (this=0x28853d0) at /home/kevin/kitware/ParaView3/VTK/Parallel/vtkPKdTree.cxx:441 #6 0x00769725 in vtkKdTreeManager::Update (this=0x2877c50) at /home/kevin/kitware/ParaView3/Servers/Filters/vtkKdTreeManager.cxx:143 #7 0x078fc403 in vtkKdTreeManagerCommand (arlu=0x24cfac0, ob=0x2877c50, method=value optimized out, m...@0x7fff6aeabc10, resultstre...@0x24ceb40) at /home/kevin/kitware/ParaView3_Build/Servers/Filters/vtkKdTreeManagerClientServer.cxx:123 #8 0x7f035f40e36f in vtkClientServerInterpreter::ProcessCommandInvoke (this=0x24cfac0, c...@0x7fff6aeac080, midx=0) at /home/kevin/kitware/ParaView3/Utilities/VTKClientServer/vtkClientServerInterpreter.cxx:384 #9 0x7f035f41058e in vtkClientServerInterpreter::ProcessOneMessage (this=0x24cfac0, c...@0x7fff6aeac080, message=0) at /home/kevin/kitware/ParaView3/Utilities/VTKClientServer/vtkClientServerInterpreter.cxx:214 #10 0x7f035f4107fd in vtkClientServerInterpreter::ProcessStream (this=0x24cfac0, c...@0x7fff6aeac080) at /home/kevin/kitware/ParaView3/Utilities/VTKClientServer/vtkClientServerInterpreter.cxx:183 #11 0x001ae220 in vtkSelfConnection::ProcessStreamLocally (this=value optimized out, stre...@0x7fff6aeac080) at /home/kevin/kitware/ParaView3/Servers/Common/vtkSelfConnection.cxx:120 #12 0x0015e7ca in vtkMPISelfConnection::SendStreamToServerNodeInternal (this=0x2985a40, remoteId=-1, stre...@0x7fff6aeac080) at /home/kevin/kitware/ParaView3/Servers/Common/vtkMPISelfConnection.cxx:246 #13 0x0015dc25 in vtkMPISelfConnection::SendStreamToDataServer (this=0x2e89fa0, stream=value optimized out) at /home/kevin/kitware/ParaView3/Servers/Common/vtkMPISelfConnection.cxx:215 ---Type return to continue, or q return to quit--- #14 0x0015fb87 in vtkProcessModuleConnection::SendStream (this=0x2985a40, servers=value optimized out, stre...@0x7fff6aeac080) at /home/kevin/kitware/ParaView3/Servers/Common/vtkProcessModuleConnection.cxx:225 #15 0x001629a9 in vtkProcessModuleConnectionManager::SendStream (this=value optimized out, connectionID=value optimized out, serverFlags=1, stre...@0x7fff6aeac080, reset=1) at /home/kevin/kitware/ParaView3/Servers/Common/vtkProcessModuleConnectionManager.cxx:536 #16 0x0016a3ca in vtkProcessModule::SendStream (this=0x24d14c0, connectionID=1, server=1, stre...@0x7fff6aeac080, resetStream=1) at /home/kevin/kitware/ParaView3/Servers/Common/vtkProcessModule.cxx:870 #17 0x0015ae85 in vtkClientConnectionRMI (remoteArg=0x2f1c3c0, remoteArgLength=value optimized out) at /home/kevin/kitware/ParaView3/Servers/Common/vtkClientConnection.cxx:57 #18 0x7f035eebef2d in vtkMultiProcessController::ProcessRMI
Re: [Paraview] Why is PKdTree used for IMAGE Volume Rendering?
Kevin, Attached is a patch. Can you verify that it works after applying the patch? If so, I'll commit it to CVS. Thanks Utkarsh On Thu, Feb 12, 2009 at 4:45 PM, Kevin H. Hobbs hob...@ohiou.edu wrote: On Thu, 2009-02-12 at 15:34 -0500, Utkarsh Ayachit wrote: Ah and there's the bad news. In case of image data for the if condition at vtkPKdTree.cxx:435, I'd expect the condition to succeed and go to this-ProcessUserDefinedCuts(..) however, it seems like that condition is failing and it's going to this-MultiProcessBuildLocator(..) which is used for unstructured datasets. Are you able to reproduce the behavior with a simpler dataset (say wavelet) or something that;s already in ParaViewData, or some dataset that I can try it with? Yes, I was able to reproduce the problem with the wavelet source data. and I got a new datum as well. It seems I have to load the large image from a .vti file for the error to occur. The error does not occur when I create a large image in parallel with the wavelet source. Also give me all the steps to reproduce the problem (since I am not able to reproduce it with 5 procs using Wavelet source). Utkarsh In paraview connected to the builtin server I created a wavelet source with extents 0 1958 0 1498 0 77 to match my file. I then saved that to a vti file. I then restarted paraview and started a 2 core server that the dialog runs with mpirun -np 2 \ /home/kevin/kitware/ParaView3_Build/bin/pvserver \ --use-offscreen-rendering I loaded the image then selected volume rendering for the representation. Then BLAMO swap-death. Index: Servers/Filters/vtkKdTreeManager.cxx === RCS file: /cvsroot/ParaView3/ParaView3/Servers/Filters/vtkKdTreeManager.cxx,v retrieving revision 1.7 diff -u -3 -p -r1.7 vtkKdTreeManager.cxx --- Servers/Filters/vtkKdTreeManager.cxx 24 Nov 2008 14:32:41 - 1.7 +++ Servers/Filters/vtkKdTreeManager.cxx 12 Feb 2009 22:24:40 - @@ -24,6 +24,7 @@ #include vtkPoints.h #include vtkPVUpdateSuppressor.h #include vtkSmartPointer.h +#include vtkSphereSource.h #include vtkUnstructuredGrid.h #define VTK_CREATE(type, name) \ @@ -38,7 +39,6 @@ class vtkKdTreeManager::vtkAlgorithmSet vtkStandardNewMacro(vtkKdTreeManager); vtkCxxRevisionMacro(vtkKdTreeManager, $Revision: 1.7 $); vtkCxxSetObjectMacro(vtkKdTreeManager, StructuredProducer, vtkAlgorithm); -vtkCxxSetObjectMacro(vtkKdTreeManager, KdTree, vtkPKdTree); // vtkKdTreeManager::vtkKdTreeManager() { @@ -46,6 +46,7 @@ vtkKdTreeManager::vtkKdTreeManager() this-StructuredProducer = 0; this-KdTree = 0; this-NumberOfPieces = 1; + this-KdTreeInitialized = false; } // @@ -95,6 +96,16 @@ void vtkKdTreeManager::RemoveAllProducer } // +void vtkKdTreeManager::SetKdTree(vtkPKdTree* tree) +{ + if (this-KdTree != tree) +{ +vtkSetObjectBodyMacro(KdTree, vtkPKdTree, tree); +this-KdTreeInitialized = false; +} +} + +// void vtkKdTreeManager::Update() { vtkAlgorithmSet::iterator iter; @@ -132,17 +143,26 @@ void vtkKdTreeManager::Update() } this-KdTree-RemoveAllDataSets(); - for (dsIter = outputs.begin(); dsIter != outputs.end(); ++dsIter) -{ -this-AddDataSetToKdTree(*dsIter); -} - - static bool initialized = false; - if (!initialized) + if (!this-KdTreeInitialized) { +// HACK: This hack fixes the following issue: +// * create wavelet (num procs = 4) +// * volume render -- broken!!! +// * change some wavelet parameter (force the KdTree to rebuild) and all's fine! +// Seems like something doesn't get initialized correctly, I have no idea +// what. This seems to overcome the issue. +vtkSphereSource* sphere = vtkSphereSource::New(); +sphere-Update(); +this-KdTree-AddDataSet(sphere-GetOutput()); +sphere-Delete(); this-KdTree-BuildLocator(); -initialized = true; +this-KdTree-RemoveAllDataSets(); +this-KdTreeInitialized = true; } + for (dsIter = outputs.begin(); dsIter != outputs.end(); ++dsIter) +{ +this-AddDataSetToKdTree(*dsIter); +} if (this-StructuredProducer) { Index: Servers/Filters/vtkKdTreeManager.h === RCS file: /cvsroot/ParaView3/ParaView3/Servers/Filters/vtkKdTreeManager.h,v retrieving revision 1.2 diff -u -3 -p -r1.2 vtkKdTreeManager.h --- Servers/Filters/vtkKdTreeManager.h 7 Sep 2007 13:34:15 - 1.2 +++ Servers/Filters/vtkKdTreeManager.h 12 Feb 2009 22:24:40 - @@ -65,6 +65,7 @@ protected: void AddDataSetToKdTree(vtkDataSet *data); + bool
Re: [Paraview] Why is PKdTree used for IMAGE Volume Rendering?
On Thu, 2009-02-12 at 17:27 -0500, Utkarsh Ayachit wrote: Kevin, Attached is a patch. Can you verify that it works after applying the patch? If so, I'll commit it to CVS. Thanks Utkarsh Yup that fixed it. The PKdTree completes immediately and then I get my volume rendering. Thank you! signature.asc Description: This is a digitally signed message part ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview