On 03/26/2010 09:55 AM, Dave Partyka wrote: > Hey Kevin, For those 5 can you run them by hand and see if you notice > why they are hanging/failing? On some of them it looks like pvserver > might be crashing? Can you attach a debugger and see if you can get a > stack trace? Thank you very much for offering to help. >
pvcrs.ExodusIIPanel.ExodusModeShapes The two data servers are spinning which reminded me of an openmpi bug so I stuck "btl_sm_num_fifos = 15" in ~/.openmpi/mca-params.conf but that didn't help. I got back traces for the render server and the two data servers: (gdb) attach 4998 [Thread debugging using libthread_db enabled] 0x00000032456d6f73 in __select_nocancel () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install e2fsprogs-libs-1.41.9-2.fc11.x86_64 expat-2.0.1-8.fc11.x86_64 fontconfig-2.8.0-1.fc11.x86_64 freetype-2.3.9-5.fc11.x86_64 glib2-2.20.5-1.fc11.x86_64 glibc-2.10.2-1.x86_64 keyutils-libs-1.2-5.fc11.x86_64 krb5-libs-1.6.3-27.fc11.x86_64 libICE-1.0.4-7.fc11.x86_64 libSM-1.1.0-4.fc11.x86_64 libX11-1.2.2-1.fc11.x86_64 libXau-1.0.4-5.fc11.x86_64 libXcursor-1.1.9-4.fc11.x86_64 libXext-1.0.99.1-3.fc11.x86_64 libXfixes-4.0.3-5.fc11.x86_64 libXi-1.2.1-1.fc11.x86_64 libXinerama-1.0.3-4.fc11.x86_64 libXrandr-1.2.99.4-3.fc11.x86_64 libXrender-0.9.4-5.fc11.x86_64 libXt-1.0.5-2.fc11.x86_64 libgcc-4.4.1-2.fc11.x86_64 libpng-1.2.43-1.fc11.x86_64 libselinux-2.0.80-1.fc11.x86_64 libstdc++-4.4.1-2.fc11.x86_64 libxcb-1.2-5.fc11.x86_64 openmpi-1.4-5.fc11.x86_64 openssl-0.9.8k-5.fc11.x86_64 python-libs-2.6-12.fc11.x86_64 qt-4.6.2-8.fc11.x86_64 qt-x11-4.6.2-8.fc11.x86_64 zlib-1.2.3-22.fc11.x86_64 (gdb) bt #0 0x00000032456d6f73 in __select_nocancel () from /lib64/libc.so.6 #1 0x00007fa5150d08cb in vtkSocket::SelectSockets(int const*, int, unsigned long, int*) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkCommon.so.pv3.7 #2 0x00007fa5150d1044 in vtkSocketCollection::SelectSockets(unsigned long) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkCommon.so.pv3.7 #3 0x00007fa51f491584 in vtkProcessModuleConnectionManager::MonitorConnections(unsigned long) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #4 0x00007fa51f497cdc in vtkProcessModule::StartServer(unsigned long) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #5 0x00007fa51f4997c7 in vtkProcessModule::Start(int, char**) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #6 0x00007fa51f09e246 in vtkPVMain::Run(vtkPVOptions*) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVFilters.so #7 0x000000000040112c in main () (gdb) detach (gdb) attach 5001 [Thread debugging using libthread_db enabled] 0x00007f19b2bd1b6b in opal_class_initialize () from /usr/lib64/openmpi/lib/openmpi/mca_btl_sm.so (gdb) bt #0 0x00007f19b2bd1b6b in opal_class_initialize () from /usr/lib64/openmpi/lib/openmpi/mca_btl_sm.so #1 0x00000033c3a123aa in opal_progress () from /usr/lib64/openmpi/lib/libopen-pal.so.0 #2 0x00000033c42363c5 in ?? () from /usr/lib64/openmpi/lib/libmpi.so.0 #3 0x00007f19b1b10b6a in opal_class_initialize () from /usr/lib64/openmpi/lib/openmpi/mca_coll_tuned.so #4 0x00007f19b1b1906f in opal_class_initialize () from /usr/lib64/openmpi/lib/openmpi/mca_coll_tuned.so #5 0x00007f19b1d2890b in opal_class_initialize () from /usr/lib64/openmpi/lib/openmpi/mca_coll_sync.so #6 0x00000033c4253f39 in PMPI_Gatherv () from /usr/lib64/openmpi/lib/libmpi.so.0 #7 0x00007f19bb55ceb7 in vtkMPICommunicator::GatherVVoidArray(void const*, void*, long long, long long*, long long*, int, int) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkParallel.so.pv3.7 #8 0x00007f19c279b111 in vtkMPIMoveData::DataServerGatherToZero(vtkDataObject*, vtkDataObject*) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVFilters.so #9 0x00007f19c279b9b6 in vtkMPIMoveData::RequestData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVFilters.so #10 0x00007f19b8d73c44 in vtkExecutive::CallAlgorithm(vtkInformation*, int, vtkInformationVector**, vtkInformationVector*) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkFiltering.so.pv3.7 #11 0x00007f19b8d667ac in vtkDemandDrivenPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkFiltering.so.pv3.7 #12 0x00007f19b8d44d01 in vtkCompositeDataPipeline::ExecuteData(vtkInformation*, vtkInformationVector**, vtkInformationVector*) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkFiltering.so.pv3.7 #13 0x00007f19b8d69819 in vtkDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkFiltering.so.pv3.7 #14 0x00007f19b8eb1751 in vtkStreamingDemandDrivenPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkFiltering.so.pv3.7 #15 0x00007f19b8d41881 in vtkCompositeDataPipeline::ProcessRequest(vtkInformation*, vtkInformationVector**, vtkInformationVector*) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkFiltering.so.pv3.7 #16 0x00007f19b8d6955f in vtkDemandDrivenPipeline::UpdateData(int) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkFiltering.so.pv3.7 #17 0x00007f19b8eb093b in vtkStreamingDemandDrivenPipeline::Update(int) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkFiltering.so.pv3.7 #18 0x00007f19c28702c9 in vtkPVUpdateSuppressor::ForceUpdate() () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVFilters.so #19 0x00007f19c1c85d90 in vtkPVUpdateSuppressorCommand(vtkClientServerInterpreter*, vtkObjectBase*, char const*, vtkClientServerStream const&, vtkClientServerStream&) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVFiltersCS.so #20 0x00007f19bd93367f in vtkClientServerInterpreter::ProcessCommandInvoke(vtkClientServerStream const&, int) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkClientServer.so #21 0x00007f19bd9344be in vtkClientServerInterpreter::ProcessOneMessage(vtkClientServerStream const&, int) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkClientServer.so #22 0x00007f19bd9345dd in vtkClientServerInterpreter::ProcessStream(vtkClientServerStream const&) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkClientServer.so #23 0x00007f19c2ca1dc0 in vtkSelfConnection::ProcessStreamLocally(vtkClientServerStream&) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #24 0x00007f19c2c440c0 in vtkMPISelfConnection::SendStreamToServerNodeInternal(int, vtkClientServerStream&) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #25 0x00007f19c2c433b5 in vtkMPISelfConnection::SendStreamToDataServer(vtkClientServerStream&) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #26 0x00007f19c2c453e7 in vtkProcessModuleConnection::SendStream(unsigned int, vtkClientServerStream&) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #27 0x00007f19c2c48146 in vtkProcessModuleConnectionManager::SendStream(long long, unsigned int, vtkClientServerStream&, int) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #28 0x00007f19c2c4febb in vtkProcessModule::SendStream(long long, unsigned int, vtkClientServerStream&, int) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #29 0x00007f19c2c406a5 in vtkClientConnectionRMI(void*, void*, int, int) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #30 0x00007f19bb498495 in vtkMultiProcessController::ProcessRMI(int, void*, int, int) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkParallel.so.pv3.7 #31 0x00007f19bb498c99 in vtkMultiProcessController::ProcessRMIs(int, int) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkParallel.so.pv3.7 #32 0x00007f19c2c9d746 in vtkRemoteConnection::ProcessCommunication() () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #33 0x00007f19c2c49665 in vtkProcessModuleConnectionManager::MonitorConnections(unsigned long) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #34 0x00007f19c2c4fcdc in vtkProcessModule::StartServer(unsigned long) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #35 0x00007f19c2c517c7 in vtkProcessModule::Start(int, char**) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #36 0x00007f19c2856246 in vtkPVMain::Run(vtkPVOptions*) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVFilters.so #37 0x000000000040112c in main () (gdb) detach (gdb) attach 5003 [Thread debugging using libthread_db enabled] 0x00007f20cdef2b59 in opal_class_initialize () from /usr/lib64/openmpi/lib/openmpi/mca_btl_sm.so (gdb) bt #0 0x00007f20cdef2b59 in opal_class_initialize () from /usr/lib64/openmpi/lib/openmpi/mca_btl_sm.so #1 0x00000033c3a123aa in opal_progress () from /usr/lib64/openmpi/lib/libopen-pal.so.0 #2 0x00007f20cf160ded in opal_class_initialize () from /usr/lib64/openmpi/lib/openmpi/mca_pml_ob1.so #3 0x00000033c425cbb7 in PMPI_Recv () from /usr/lib64/openmpi/lib/libmpi.so.0 #4 0x00007f20d687bb40 in vtkMPICommunicator::ReceiveDataInternal(char*, int, int, int, int, vtkMPICommunicatorReceiveDataInfo*, int, int&) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkParallel.so.pv3.7 #5 0x00007f20d687e649 in vtkMPICommunicator::ReceiveVoidArray(void*, long long, int, int, int) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkParallel.so.pv3.7 #6 0x00007f20d67b9bde in vtkMultiProcessController::ProcessRMIs(int, int) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkParallel.so.pv3.7 #7 0x00007f20ddf72739 in vtkProcessModule::Start(int, char**) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVServerCommon.so #8 0x00007f20ddb77246 in vtkPVMain::Run(vtkPVOptions*) () from /home/kevin/kitware/ParaView3_Build/bin/libvtkPVFilters.so #9 0x000000000040112c in main () (gdb) detach
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
