Hello all,
I have 2 rather simple questions and have been scratching my head
for the last hour for an answer. Probably just need a fresh pair of eyes!.
My MPI lib is Intel MPI 3.2.1
The code segment that has been the bane of my last hour of existence is :
if (MPI_Bcast(ssx,tp, MPI_INT, 0, cartcomm) != MPI_SUCCESS)
printf("Error broadcasting\n");
if (MPI_Bcast(ssy,tp, MPI_INT, 0, cartcomm) != MPI_SUCCESS)
printf("Error broadcasting\n");
if (MPI_Bcast(ssz,tp, MPI_INT, 0, cartcomm) != MPI_SUCCESS)
printf("Error broadcasting\n");
if (MPI_Bcast(eex,tp, MPI_INT, 0, cartcomm) != MPI_SUCCESS)
printf("Error broadcasting\n");
if (MPI_Bcast(eey,tp, MPI_INT, 0, cartcomm) != MPI_SUCCESS)
printf("Error broadcasting\n");
if (MPI_Bcast(eez,tp, MPI_INT, 0, cartcomm) != MPI_SUCCESS)
printf("Error broadcasting\n");
if (myrank != 0) {
MPI_Recv(data2, (int) (sizeof(data2)/sizeof(short)),
MPI_SHORT, 0, 123, cartcomm, &status[0]);
}
else {
short data[dim_siz[0]][dim_siz[1]][dim_siz[2]];
if (nc_get_var_short(ncid, varid, &data[0][0][0]) !=
NC_NOERR)
printf("Error 2");
rank = 1;
while (rank <= numprocs-1) {
for (l=ssz[rank]; l<eez[rank]; l+=1)
for (m=ssy[rank]; m<eey[rank]; m+=1)
for (n=ssx[rank]; n<eex[rank]; n+=1)
{
data2[l-ssz[rank]][m-ssy[rank]][n-ssx[rank]] = data[l][m][n];
}
MPI_Isend(&data2[0][0][0], (int)
(sizeof(data2)/sizeof(short)), MPI_SHORT, rank, 123,
cartcomm,&request[rank-1]);
printf("My rank is %d and my ending value is %d on
Data2 and %d on Data\n", rank,
data2[l-ssz[rank]-1][m-ssy[rank]-1][n-ssx[rank]-1], data[l-1][m-1][n-1]);
printf("My rank is %d and my starting value is %d on
Data2 and %d on Data\n", rank, data2[0][0][0],
data[ssz[rank]][ssy[rank]][ssx[rank]]);
rank++;
}
MPI_Waitall(numprocs-1, request, status);
}
printf("My rank is %d and my starting value is %d on Data2\n",
myrank, data2[0][0][0]);
printf("My rank is %d and my ending value is %d on Data2\n", myrank,
data2[l-ssz[myrank]-1][m-ssy[myrank]-1][n-ssx[myrank]-1]);
cartcomm is the cartesian grid communicator and size of data is 512^3.
The objective of this part of the code is to distribute data in a
checkerboard pattern to np procs. Unfortunately I am running into 2
complications.
1) When I use MPI_SHORT instead of short to count the number of data
transferred I get an array that is half the size. (ie
MPI_Isend(&data2[0][0][0],
(int) (sizeof(data2)/sizeof(MPI_SHORT)), MPI_SHORT, rank, 123,
cartcomm,&request[rank-1]);
So is there any possible way MPI_SHORT can be set to 32 bit? I have never
come across this before and was stumped to notice! Maybe my ignorance comes
from having only installed openmpi on my personal system?
2) The output from this part of the code segment, for eg. when run on 8
procs, looks like all the ranks from 1 - 7 have exactly the same array and
the array should actually only belong to rank 7. .. the op spewed out by gdb
is below :
0: My rank is 1 and my ending value is 3277 on Data2 and 3277 on Data
0: My rank is 1 and my starting value is 3277 on Data2 and 3277 on Data
0: My rank is 2 and my ending value is 3277 on Data2 and 3277 on Data
0: My rank is 2 and my starting value is 3277 on Data2 and 3277 on Data
0: My rank is 3 and my ending value is 3277 on Data2 and 3277 on Data
0: My rank is 3 and my starting value is 3277 on Data2 and 3277 on Data
0: My rank is 4 and my ending value is 3277 on Data2 and 3277 on Data
0: My rank is 4 and my starting value is 3277 on Data2 and 3277 on Data
0: My rank is 5 and my ending value is 3277 on Data2 and 3277 on Data
0: My rank is 5 and my starting value is 3277 on Data2 and 3277 on Data
0: My rank is 6 and my ending value is 3277 on Data2 and 3277 on Data
0: My rank is 6 and my starting value is 3277 on Data2 and 3277 on Data
0: *My rank is 7 and my ending value is 3277 on Data2 and 3277 on Data*
0: *My rank is 7 and my starting value is 3866 on Data2 and 3866 on Data*
1: My rank is 1 and my starting value is 3866 on Data2
2: My rank is 2 and my starting value is 3866 on Data2
3: My rank is 3 and my starting value is 3866 on Data2
1: My rank is 1 and my ending value is 3277 on Data2
2: My rank is 2 and my ending value is 3277 on Data2
3: My rank is 3 and my ending value is 3277 on Data2
7: My rank is 7 and my starting value is 3866 on Data2
7: My rank is 7 and my ending value is 3277 on Data2
5: My rank is 5 and my starting value is 3866 on Data2
6: My rank is 6 and my starting value is 3866 on Data2
6: My rank is 6 and my ending value is 3277 on Data2
0: My rank is 0 and my starting value is 3866 on Data2
1-3,6-7: (gdb) 4: My rank is 4 and my starting value is 3866 on Data2
5: My rank is 5 and my ending value is 3277 on Data2
1-3,6-7: (gdb) 0: My rank is 0 and my ending value is 3277 on Data2
4: My rank is 4 and my ending value is 3277 on Data2
I was wondering if someone might be able to point out what I am doing wrong
here that only the last block of the array is getting transmitted to all
procs?
My apologies for the vagueness of the question. Appreciate any help!
Cheers and thanks in advance,
C.S.N
