[Paraview] Why is PKdTree used for IMAGE Volume Rendering?

2009-02-12 Thread Kevin H. Hobbs 
I just tried to use ParaView from Tuesday's CVS (The network was down
last night.) on both CPU cores to volume render a large (1/4 the size of
available RAM) image.

As soon as I selected volume render for the representation I saw the
status bar show PKdTree and moments later all of the system memory was
consumed and the system started swapping.

I had to kill paraview and the pvserver processes from the console to
end the swap-death.

I reran paraview with only one CPU core and proceeded as normal.

My question is: why is PKdTree needed at all for image data?

For unstructured data it of course makes sense but for image data it
just wastes a lot of RAM.

I haven't tried parallel image volume rendering in a while so I don't
know when this showed up.



signature.asc
Description: This is a digitally signed message part
___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: 
http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview

Re: [Paraview] Why is PKdTree used for IMAGE Volume Rendering?

2009-02-12 Thread Utkarsh Ayachit 
Do you have any unstructured data being rendering in  the scene?
KdTree is used to distribute that unstructured data in accordance with
the partitions provided by the structured data.

Utkarsh

On Thu, Feb 12, 2009 at 2:09 PM, Kevin H. Hobbs hob...@ohiou.edu wrote:
 I just tried to use ParaView from Tuesday's CVS (The network was down
 last night.) on both CPU cores to volume render a large (1/4 the size of
 available RAM) image.

 As soon as I selected volume render for the representation I saw the
 status bar show PKdTree and moments later all of the system memory was
 consumed and the system started swapping.

 I had to kill paraview and the pvserver processes from the console to
 end the swap-death.

 I reran paraview with only one CPU core and proceeded as normal.

 My question is: why is PKdTree needed at all for image data?

 For unstructured data it of course makes sense but for image data it
 just wastes a lot of RAM.

 I haven't tried parallel image volume rendering in a while so I don't
 know when this showed up.


 ___
 Powered by www.kitware.com

 Visit other Kitware open-source projects at 
 http://www.kitware.com/opensource/opensource.html

 Please keep messages on-topic and check the ParaView Wiki at: 
 http://paraview.org/Wiki/ParaView

 Follow this link to subscribe/unsubscribe:
 http://www.paraview.org/mailman/listinfo/paraview


___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: 
http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview

Re: [Paraview] Why is PKdTree used for IMAGE Volume Rendering?

2009-02-12 Thread Kevin H. Hobbs 
On Thu, 2009-02-12 at 14:26 -0500, Utkarsh Ayachit wrote:
 Do you have any unstructured data being rendering in  the scene?

No, just one big .vti image.


signature.asc
Description: This is a digitally signed message part
___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: 
http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview

Re: [Paraview] Why is PKdTree used for IMAGE Volume Rendering?

2009-02-12 Thread Utkarsh Ayachit 
Kevin,

The KdTree is still built (using the partitions from the structured
dataset set itself -- hence isn't a very compute intensive task) just
in case it's needed to partition any unstructured data, hence you're
seeing the PKdTree progress. But I don't think that's where it is
hanging. Is the progress bar saying PKtree:100 ? It may actually have
frozen in the volume rendering code. Is it possible to attach a
debugger to any of the server processes and see where it's stuck
(provide the stack trace if possible)?

Utkarsh

On Thu, Feb 12, 2009 at 2:37 PM, Kevin H. Hobbs hob...@ohiou.edu wrote:
 On Thu, 2009-02-12 at 14:26 -0500, Utkarsh Ayachit wrote:
 Do you have any unstructured data being rendering in  the scene?

 No, just one big .vti image.

___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: 
http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview

Re: [Paraview] Why is PKdTree used for IMAGE Volume Rendering?

2009-02-12 Thread Utkarsh Ayachit 
Ah and there's the bad news.
In case of image data for the if condition at vtkPKdTree.cxx:435,
I'd expect the condition to succeed and go to
this-ProcessUserDefinedCuts(..) however, it seems like that condition
is failing and it's going to this-MultiProcessBuildLocator(..) which
is used for unstructured datasets.

Are you able to reproduce the behavior with a simpler dataset (say
wavelet) or something that;s already in ParaViewData, or some dataset
that I can try it with? Also give me all the steps to reproduce the
problem (since I am not able to reproduce it with 5 procs using
Wavelet source).

Utkarsh


On Thu, Feb 12, 2009 at 3:24 PM, Kevin H. Hobbs hob...@ohiou.edu wrote:
 On Thu, 2009-02-12 at 14:56 -0500, Utkarsh Ayachit wrote:
 Kevin,

 The KdTree is still built (using the partitions from the structured
 dataset set itself -- hence isn't a very compute intensive task) just
 in case it's needed to partition any unstructured data, hence you're
 seeing the PKdTree progress. But I don't think that's where it is
 hanging. Is the progress bar saying PKtree:100 ?

 No, it only made it to 58

  It may actually have
 frozen in the volume rendering code. Is it possible to attach a
 debugger to any of the server processes and see where it's stuck
 (provide the stack trace if possible)?

 Paraview stops completely as soon as I attach but here is the first bt
 before it really hung up:

 (gdb) bt
 #0  vtkDataArrayTemplatedouble::DataChanged (this=0x2e89fa0)
at /home/kevin/kitware/ParaView3/VTK/Common/vtkDataArrayTemplate.txx:1202
 #1  0x7f035a9cc221 in vtkPoints::SetPoint () at 
 /home/kevin/kitware/ParaView3/VTK/Common/vtkPoints.h:126
 #2  vtkImageData::GetCell (this=0x2e897d0, cellId=48799904)
at /home/kevin/kitware/ParaView3/VTK/Filtering/vtkImageData.cxx:392
 #3  0x7f035aa00bed in vtkKdTree::ComputeCellCenters (this=0x28853d0, 
 set=0x0)
at /home/kevin/kitware/ParaView3/VTK/Filtering/vtkKdTree.cxx:677
 #4  0x7f035eee71eb in vtkPKdTree::MultiProcessBuildLocator 
 (this=0x28853d0, volBounds=0x2e91140)
at /home/kevin/kitware/ParaView3/VTK/Parallel/vtkPKdTree.cxx:498
 #5  0x7f035eee7683 in vtkPKdTree::BuildLocator (this=0x28853d0)
at /home/kevin/kitware/ParaView3/VTK/Parallel/vtkPKdTree.cxx:441
 #6  0x00769725 in vtkKdTreeManager::Update (this=0x2877c50)
at /home/kevin/kitware/ParaView3/Servers/Filters/vtkKdTreeManager.cxx:143
 #7  0x078fc403 in vtkKdTreeManagerCommand (arlu=0x24cfac0, 
 ob=0x2877c50, method=value optimized out,
m...@0x7fff6aeabc10, resultstre...@0x24ceb40)
at 
 /home/kevin/kitware/ParaView3_Build/Servers/Filters/vtkKdTreeManagerClientServer.cxx:123
 #8  0x7f035f40e36f in vtkClientServerInterpreter::ProcessCommandInvoke 
 (this=0x24cfac0, c...@0x7fff6aeac080,
midx=0) at 
 /home/kevin/kitware/ParaView3/Utilities/VTKClientServer/vtkClientServerInterpreter.cxx:384
 #9  0x7f035f41058e in vtkClientServerInterpreter::ProcessOneMessage 
 (this=0x24cfac0, c...@0x7fff6aeac080,
message=0) at 
 /home/kevin/kitware/ParaView3/Utilities/VTKClientServer/vtkClientServerInterpreter.cxx:214
 #10 0x7f035f4107fd in vtkClientServerInterpreter::ProcessStream 
 (this=0x24cfac0, c...@0x7fff6aeac080)
at 
 /home/kevin/kitware/ParaView3/Utilities/VTKClientServer/vtkClientServerInterpreter.cxx:183
 #11 0x001ae220 in vtkSelfConnection::ProcessStreamLocally 
 (this=value optimized out,
stre...@0x7fff6aeac080) at 
 /home/kevin/kitware/ParaView3/Servers/Common/vtkSelfConnection.cxx:120
 #12 0x0015e7ca in 
 vtkMPISelfConnection::SendStreamToServerNodeInternal (this=0x2985a40, 
 remoteId=-1,
stre...@0x7fff6aeac080) at 
 /home/kevin/kitware/ParaView3/Servers/Common/vtkMPISelfConnection.cxx:246
 #13 0x0015dc25 in vtkMPISelfConnection::SendStreamToDataServer 
 (this=0x2e89fa0,
stream=value optimized out) at 
 /home/kevin/kitware/ParaView3/Servers/Common/vtkMPISelfConnection.cxx:215
 ---Type return to continue, or q return to quit---
 #14 0x0015fb87 in vtkProcessModuleConnection::SendStream 
 (this=0x2985a40, servers=value optimized out,
stre...@0x7fff6aeac080) at 
 /home/kevin/kitware/ParaView3/Servers/Common/vtkProcessModuleConnection.cxx:225
 #15 0x001629a9 in vtkProcessModuleConnectionManager::SendStream 
 (this=value optimized out,
connectionID=value optimized out, serverFlags=1, stre...@0x7fff6aeac080, 
 reset=1)
at 
 /home/kevin/kitware/ParaView3/Servers/Common/vtkProcessModuleConnectionManager.cxx:536
 #16 0x0016a3ca in vtkProcessModule::SendStream (this=0x24d14c0, 
 connectionID=1, server=1,
stre...@0x7fff6aeac080, resetStream=1) at 
 /home/kevin/kitware/ParaView3/Servers/Common/vtkProcessModule.cxx:870
 #17 0x0015ae85 in vtkClientConnectionRMI (remoteArg=0x2f1c3c0, 
 remoteArgLength=value optimized out)
at /home/kevin/kitware/ParaView3/Servers/Common/vtkClientConnection.cxx:57
 #18 0x7f035eebef2d in vtkMultiProcessController::ProcessRMI

Re: [Paraview] Why is PKdTree used for IMAGE Volume Rendering?

2009-02-12 Thread Utkarsh Ayachit 
Kevin,

Attached is a patch. Can you verify that it works after applying the
patch? If so, I'll commit it to CVS.

Thanks
Utkarsh

On Thu, Feb 12, 2009 at 4:45 PM, Kevin H. Hobbs hob...@ohiou.edu wrote:
 On Thu, 2009-02-12 at 15:34 -0500, Utkarsh Ayachit wrote:
 Ah and there's the bad news.
 In case of image data for the if condition at vtkPKdTree.cxx:435,
 I'd expect the condition to succeed and go to
 this-ProcessUserDefinedCuts(..) however, it seems like that condition
 is failing and it's going to this-MultiProcessBuildLocator(..) which
 is used for unstructured datasets.

 Are you able to reproduce the behavior with a simpler dataset (say
 wavelet) or something that;s already in ParaViewData, or some dataset
 that I can try it with?

 Yes, I was able to reproduce the problem with the wavelet source data.
 and I got a new datum as well.

 It seems I have to load the large image from a .vti file for the error
 to occur.

 The error does not occur when I create a large image in parallel with
 the wavelet source.

  Also give me all the steps to reproduce the
 problem (since I am not able to reproduce it with 5 procs using
 Wavelet source).

 Utkarsh

 In paraview connected to the builtin server I created a wavelet source
 with extents 0 1958 0 1498 0 77 to match my file.

 I then saved that to a vti file.

 I then restarted paraview and started a 2 core server that the dialog
 runs with

mpirun -np 2 \
/home/kevin/kitware/ParaView3_Build/bin/pvserver \
--use-offscreen-rendering

 I loaded the image then selected volume rendering for the
 representation.

 Then BLAMO swap-death.



Index: Servers/Filters/vtkKdTreeManager.cxx
===
RCS file: /cvsroot/ParaView3/ParaView3/Servers/Filters/vtkKdTreeManager.cxx,v
retrieving revision 1.7
diff -u -3 -p -r1.7 vtkKdTreeManager.cxx
--- Servers/Filters/vtkKdTreeManager.cxx	24 Nov 2008 14:32:41 -	1.7
+++ Servers/Filters/vtkKdTreeManager.cxx	12 Feb 2009 22:24:40 -
@@ -24,6 +24,7 @@
 #include vtkPoints.h
 #include vtkPVUpdateSuppressor.h
 #include vtkSmartPointer.h
+#include vtkSphereSource.h
 #include vtkUnstructuredGrid.h
 
 #define VTK_CREATE(type, name) \
@@ -38,7 +39,6 @@ class vtkKdTreeManager::vtkAlgorithmSet 
 vtkStandardNewMacro(vtkKdTreeManager);
 vtkCxxRevisionMacro(vtkKdTreeManager, $Revision: 1.7 $);
 vtkCxxSetObjectMacro(vtkKdTreeManager, StructuredProducer, vtkAlgorithm);
-vtkCxxSetObjectMacro(vtkKdTreeManager, KdTree, vtkPKdTree);
 //
 vtkKdTreeManager::vtkKdTreeManager()
 {
@@ -46,6 +46,7 @@ vtkKdTreeManager::vtkKdTreeManager()
   this-StructuredProducer = 0;
   this-KdTree = 0;
   this-NumberOfPieces = 1;
+  this-KdTreeInitialized = false;
 }
 
 //
@@ -95,6 +96,16 @@ void vtkKdTreeManager::RemoveAllProducer
 }
 
 //
+void vtkKdTreeManager::SetKdTree(vtkPKdTree* tree)
+{
+  if (this-KdTree != tree)
+{
+vtkSetObjectBodyMacro(KdTree, vtkPKdTree, tree);
+this-KdTreeInitialized = false;
+}
+}
+
+//
 void vtkKdTreeManager::Update()
 {
   vtkAlgorithmSet::iterator iter;
@@ -132,17 +143,26 @@ void vtkKdTreeManager::Update()
 }
 
   this-KdTree-RemoveAllDataSets();
-  for (dsIter = outputs.begin(); dsIter != outputs.end(); ++dsIter)
-{
-this-AddDataSetToKdTree(*dsIter);
-}
-
-  static bool initialized = false;
-  if (!initialized)
+  if (!this-KdTreeInitialized)
 {
+// HACK: This hack fixes the following issue:
+// * create wavelet (num procs = 4)
+// * volume render -- broken!!!
+// * change some wavelet parameter (force the KdTree to rebuild) and all's fine!
+// Seems like something doesn't get initialized correctly, I have no idea
+// what. This seems to overcome the issue.
+vtkSphereSource* sphere = vtkSphereSource::New();
+sphere-Update();
+this-KdTree-AddDataSet(sphere-GetOutput());
+sphere-Delete();
 this-KdTree-BuildLocator();
-initialized = true;
+this-KdTree-RemoveAllDataSets();
+this-KdTreeInitialized = true;
 }
+  for (dsIter = outputs.begin(); dsIter != outputs.end(); ++dsIter)
+{
+this-AddDataSetToKdTree(*dsIter);
+} 
 
   if (this-StructuredProducer)
 {
Index: Servers/Filters/vtkKdTreeManager.h
===
RCS file: /cvsroot/ParaView3/ParaView3/Servers/Filters/vtkKdTreeManager.h,v
retrieving revision 1.2
diff -u -3 -p -r1.2 vtkKdTreeManager.h
--- Servers/Filters/vtkKdTreeManager.h	7 Sep 2007 13:34:15 -	1.2
+++ Servers/Filters/vtkKdTreeManager.h	12 Feb 2009 22:24:40 -
@@ -65,6 +65,7 @@ protected:
 
   void AddDataSetToKdTree(vtkDataSet *data);
 
+  bool

Re: [Paraview] Why is PKdTree used for IMAGE Volume Rendering?

2009-02-12 Thread Kevin H. Hobbs 
On Thu, 2009-02-12 at 17:27 -0500, Utkarsh Ayachit wrote:
 Kevin,
 
 Attached is a patch. Can you verify that it works after applying the
 patch? If so, I'll commit it to CVS.
 
 Thanks
 Utkarsh

Yup that fixed it. The PKdTree completes immediately and then I get my
volume rendering.

Thank you!


signature.asc
Description: This is a digitally signed message part
___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: 
http://paraview.org/Wiki/ParaView

Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview