mp_sum_rt is interfaced with mp_sum for 3d array. When you mp_sum a 3d array, it will be automatically called. USE mp, ONLY : mp_sum_rt is not necessary. If you want to check if mp_sum_rt is implicitly called via interface, you can print something before and after your mp_sum call and also print something in mp_sum_rt. You should find the expected printing behaviour.
if you compare the mp_sum_rt with the 2d and 4d implementation mp_sum_rm and mp_sum_r4d, you should find they flat the dimensionality and call the same 1-d routine reduce_base_real. As I asked earlier, how did you allocate your 3-D arrays input and output? Are you allocating sufficient size for the output, did you deallocate it by mistake? How large is output array size, if it exceeds 2^32, you might hit the 32bit integer bug of the variable msglen. Ye =================== Ye Luo, Ph.D. Leadership Computing Facility Argonne National Laboratory 2016-10-30 14:25 GMT-05:00 Vahid Askarpour <vh261...@dal.ca>: > Hi Ye, > > The changes I am making are part of the EPW/QE code and so the > parallelization depends on the QE MPI routines. I have no problem using > mp_sum for a 2-D array such as > energy(iband,ik). I can use mp_sum to add the contributions from various > nodes and print the total in agreement with the output from the serial run. > As far as the allocation goes, I use the same 2-D approach to my 3-D array. > > The problem arises when I use the 3-D array with mp_sum. I noticed that in > the Modules/mp.f90 file, there is a subroutine as follows: > > SUBROUTINE mp_sum_rt( msg, gid ) > IMPLICIT NONE > REAL (DP), INTENT (INOUT) :: msg(:,:,:) > INTEGER, INTENT(IN) :: gid > #if defined(__MPI) > INTEGER :: msglen > msglen = size(msg) > CALL reduce_base_real( msglen, msg, gid, -1 ) > #endif > END SUBROUTINE mp_sum_rt > > > Here msg has three dimensions. There are other subroutines for 1, 2, and > 4 dimensions. Somehow, if I can get EPW to use the above subroutine, it > might work. Adding a statement like: > > USE mp, ONLY : mp_sum_rt > > at the beginning of the file does not work as the compilation fails > because EPW does not see the subroutine. Interestingly, it does see mp_sum > which is in the same mp.f90 file. > > So is there a way that I can get EPW to see mp_sum_rt? This might solve my > problem. > > Since I know that mp_sum works with 2-D arrays, one option is to rewrite > my files in 2-D, one array for each x,y,z direction. I am trying to avoid > this. > > Thanks, > > Vahid > > > On Oct 30, 2016, at 3:59 PM, Ye Luo <xw111lu...@gmail.com> wrote: > > Hi Vahid, > segfault in mp_sum doesn't necessarily mean a problem there. Probably you > wrote something to output array but not in a valid place before mp_sum. > Try to check your allocation of output and the copy make sure they are > correct. > > Ye > > > =================== > Ye Luo, Ph.D. > Leadership Computing Facility > Argonne National Laboratory > > 2016-10-28 14:33 GMT-05:00 Vahid Askarpour <vh261...@dal.ca>: > >> Hi Ye, >> >> Thank you for your suggestion. I tried it and when I ran the code, it >> seg-faulted. I put flags in the code to see where the segmentation faults >> occurs. It happens as the code calls mp_sum. It seems that mp_sum may not >> be able to handle this reduction. >> >> Cheers, >> Vahid >> >> On Oct 28, 2016, at 2:51 PM, Ye Luo <xw111lu...@gmail.com> wrote: >> >> In Fortran, whatever-D array is 1-D array. mp_sum should be fine. >> I saw something strange in your code that you were not copying the right >> things as you expected. >> How about the following? >> output(1:3,1:nbnds,(k_pool*pool_id+1:k_pool*pool_id+k_pool)) >> =input(1:3,1:nbnds,1:k_pool) >> >> Ye >> >> =================== >> Ye Luo, Ph.D. >> Leadership Computing Facility >> Argonne National Laboratory >> >> 2016-10-28 12:29 GMT-05:00 Vahid Askarpour <vh261...@dal.ca>: >> >>> Dear QE Users, >>> >>> I am working on some modifications to the QE-6.0 code using symmetry. >>> When I try to combine a 3-D array scattered across nodes, I use the >>> following: >>> >>> output(3,nbnds,(k_pool*pool_id+1:k_pool*pool_id+k_pool))=inp >>> ut(3,nbnds,1:k_pool) >>> >>> Here, nbnds is the number of bands, k_pool is the number of k >>> points/pool, and pool_id is the id of the pool. Here I am assuming the the >>> number of k points is divisible by the number of pools. >>> >>> Then I call mp_sum(output,inter_pool_comm) to put all the segments of >>> input across the nodes into one output file. >>> >>> When I run the modified QE code in parallel, the output file is >>> different from the serial run. >>> >>> Does the QE's mp_sum allow the above operation for a three-D array? >>> >>> Any hints or suggestions would be greatly appreciated. >>> >>> Vahid >>> >>> Vahid Askarpour >>> Department of Physics and Atmospheric Science >>> Dalhousie University, >>> Halifax, NS, Canada >>> _______________________________________________ >>> Pw_forum mailing list >>> Pw_forum@pwscf.org >>> http://pwscf.org/mailman/listinfo/pw_forum >>> >> >> _______________________________________________ >> Pw_forum mailing list >> Pw_forum@pwscf.org >> http://pwscf.org/mailman/listinfo/pw_forum >> >> >> >> _______________________________________________ >> Pw_forum mailing list >> Pw_forum@pwscf.org >> http://pwscf.org/mailman/listinfo/pw_forum >> > > _______________________________________________ > Pw_forum mailing list > Pw_forum@pwscf.org > http://pwscf.org/mailman/listinfo/pw_forum > > > > _______________________________________________ > Pw_forum mailing list > Pw_forum@pwscf.org > http://pwscf.org/mailman/listinfo/pw_forum >
_______________________________________________ Pw_forum mailing list Pw_forum@pwscf.org http://pwscf.org/mailman/listinfo/pw_forum