On Mon, Dec 3, 2012 at 1:23 PM, Fande Kong <fd.kong at siat.ac.cn> wrote: > Are there any constraints for IS and Vec?
No, but this appears to be inconsistency. Matt > On Mon, Dec 3, 2012 at 12:19 PM, Barry Smith <bsmith at mcs.anl.gov> wrote: >> >> >> Perhaps some bad data is being passed into VecScatterCreate(). I would >> suggest having SpmcsSFCreateVecScatter >> validate the IS's and Vecs being passed in. For example, do the IS have >> tons of duplicates, how long are they etc? >> >> Barry >> >> On Dec 3, 2012, at 1:12 PM, Fande Kong <fd.kong at siat.ac.cn> wrote: >> >> > More details for the errors: >> > >> > [0]PETSC ERROR: PetscGatherMessageLengths() line 133 in >> > src/sys/utils/mpimesg.c >> > [0]PETSC ERROR: VecScatterCreate_PtoP() line 2188 in >> > src/vec/vec/utils/vpscat.c >> > [0]PETSC ERROR: VecScatterCreate() line 1431 in >> > src/vec/vec/utils/vscat.c >> > [0]PETSC ERROR: SpmcsSFCreateVecScatter() line 96 in SpmcsSFComm.cpp >> > [0]PETSC ERROR: moveDataBetweenRootsAndLeaves() line 133 in >> > SpmcsSFComm.cpp >> > [0]PETSC ERROR: SpmcsSFCreateNormalizedEmbeddedSF() line 359 in >> > SpmcsSFComm.cpp >> > [0]PETSC ERROR: SpmcsSFDistributeSection() line 343 in SpmcsSection.cpp >> > [0]PETSC ERROR: SpmcsMeshDistribute() line 444 in distributeMesh.cpp >> > [0]PETSC ERROR: DMmeshInitialize() line 32 in mgInitialize.cpp >> > [0]PETSC ERROR: main() line 64 in linearElasticity3d.cpp >> > application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0 >> > >> > >> > ===================================================================================== >> > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES >> > = EXIT CODE: 256 >> > = CLEANING UP REMAINING PROCESSES >> > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES >> > >> > ===================================================================================== >> > [proxy:0:1 at node1778] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:1 at node1778] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:1 at node1778] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:2 at node1777] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:2 at node1777] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:2 at node1777] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:3 at node1773] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:3 at node1773] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:3 at node1773] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:4 at node1770] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:4 at node1770] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:4 at node1770] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:6 at node1760] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:6 at node1760] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:6 at node1760] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:7 at node1758] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:7 at node1758] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:7 at node1758] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:8 at node1738] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:8 at node1738] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:8 at node1738] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:9 at node1736] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:9 at node1736] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:9 at node1736] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:10 at node1668] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:10 at node1668] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:10 at node1668] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:11 at node1667] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:11 at node1667] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:11 at node1667] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:12 at node1658] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:12 at node1658] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:12 at node1658] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:13 at node1656] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:13 at node1656] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:13 at node1656] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:14 at node1637] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:14 at node1637] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:14 at node1637] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:15 at node1636] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:15 at node1636] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:15 at node1636] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:16 at node1611] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:16 at node1611] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:16 at node1611] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:17 at node1380] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:17 at node1380] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:17 at node1380] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:18 at node1379] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:18 at node1379] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:18 at node1379] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:19 at node1378] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:19 at node1378] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:19 at node1378] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:20 at node1377] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:20 at node1377] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:20 at node1377] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:21 at node1376] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:21 at node1376] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:21 at node1376] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:22 at node1375] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:22 at node1375] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:22 at node1375] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:23 at node1374] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:23 at node1374] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:23 at node1374] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:24 at node1373] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:24 at node1373] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:24 at node1373] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:25 at node1372] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:25 at node1372] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:25 at node1372] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:26 at node1371] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:26 at node1371] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:26 at node1371] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:27 at node1370] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:27 at node1370] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:27 at node1370] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:28 at node1369] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:28 at node1369] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:28 at node1369] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:29 at node1368] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:29 at node1368] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:29 at node1368] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:30 at node1367] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:30 at node1367] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:30 at node1367] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [proxy:0:31 at node1366] HYD_pmcd_pmip_control_cmd_cb >> > (./pm/pmiserv/pmip_cb.c:906): assert (!closed) failed >> > [proxy:0:31 at node1366] HYDT_dmxu_poll_wait_for_event >> > (./tools/demux/demux_poll.c:77): callback returned error status >> > [proxy:0:31 at node1366] main (./pm/pmiserv/pmip.c:214): demux engine error >> > waiting for event >> > [mpiexec at node1780] HYDT_bscu_wait_for_completion >> > (./tools/bootstrap/utils/bscu_wait.c:70): one of the processes terminated >> > badly; aborting >> > [mpiexec at node1780] HYDT_bsci_wait_for_completion >> > (./tools/bootstrap/src/bsci_wait.c:23): launcher returned error waiting for >> > completion >> > [mpiexec at node1780] HYD_pmci_wait_for_completion >> > (./pm/pmiserv/pmiserv_pmci.c:199): launcher returned error waiting for >> > completion >> > [mpiexec at node1780] main (./ui/mpich/mpiexec.c:385): process manager >> > error waiting for completion >> > >> > It seems nothing. >> > >> > On Mon, Dec 3, 2012 at 11:41 AM, Matthew Knepley <knepley at gmail.com> >> > wrote: >> > On Mon, Dec 3, 2012 at 12:38 PM, Fande Kong <fande.kong at colorado.edu> >> > wrote: >> > > Hi all, >> > > >> > > Can anyone guess the possible reason of the following errors: >> > > >> > > >> > > [0]PETSC ERROR: PetscGatherMessageLengths() line 133 in >> > > src/sys/utils/mpimesg.c >> > > [0]PETSC ERROR: VecScatterCreate_PtoP() line 2188 in >> > > src/vec/vec/utils/vpscat.c >> > > [0]PETSC ERROR: VecScatterCreate() line 1431 in >> > > src/vec/vec/utils/vscat.c >> > >> > Partial error messages are generally not helpful. >> > >> > Matt >> > >> > > I have been working for several days to figure out the reason, but now >> > > I >> > > still get nothing. I use Petsc-3.3-p3 based on the mvapich2-1.6. I >> > > tried to >> > > use vecscatter to distribute the mesh. When the mesh was small, >> > > everything >> > > was ok. But when the mesh became larger about 14,000,000 elements, I >> > > got the >> > > above errors. >> > > >> > > -- >> > > Fande Kong >> > > Department of Computer Science >> > > University of Colorado at Boulder >> > > >> > > >> > >> > >> > >> > -- >> > What most experimenters take for granted before they begin their >> > experiments is infinitely more interesting than any results to which >> > their experiments lead. >> > -- Norbert Wiener >> > >> > >> > >> > >> > -- >> > Fande Kong >> > ShenZhen Institutes of Advanced Technology >> > Chinese Academy of Sciences >> > >> >> > > > > -- > Fande Kong > ShenZhen Institutes of Advanced Technology > Chinese Academy of Sciences > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
