Dear all,I've found an issue when running meep-mpi 0.20.3 in our cluster: when I submit a job to the queue using a number of processes that is not a power of two, the h5 eps and the results are weird.
This does not happen when running meep-mpi through mpirun in a single node, for any number of processes. This does not happen using the regular non-mpi version of meep.
I attach the ctl and shell script used to send the jobs to the cluster. We compiled meep against openmpi instead of mpich (pbs queue issues).I attach eps png files, two are 'ok', one using the cluster queue with 4 nodes and 4 processes (qsub sphere.sh) and the other using mpi-run on a single node (mpirun -np 3 meep-mpi resolution=40 sphere.ctl). There's one giving wrong results using 5 nodes and 10 processes (through qsub sphere.sh too).
All your suggestions are welcomed. Thanks in advance. -- Dr. Pascual Muñoz Muñoz - Telecommunication Engineer Optical Communications Group - iTEAM Research Institute [email protected] - www.gco.upv.es - www.upv.es -------------------------------------------------------
sphere.sh
Description: Bourne shell script
<<inline: sphere-eps-000000.00-nodes-1-processes-3-mpi-run.png>>
<<inline: sphere-eps-000000.00-nodes-5-processes-10-qsub.png>>
<<inline: sphere-eps-000000.00-nodes-4-processes-4-qsub.png>>
(define-param n 3.5); Core refractive index
(define-param r 1); Sphere radius
(define-param pad 1) ; Padding between waveguide and edge of PML
(define-param dpml 1) ; Thickness of PML
(define sr (+ (* 2 r) pad dpml)) ; radial size (cell is from 0 to sr)
(set! geometry-lattice (make lattice (size sr sr no-size)))
(define silicon (make dielectric (index n)))
(set! geometry
(list
(make sphere (center 0 0 0) (radius r) (material silicon))
)
)
(set! pml-layers (list (make pml (thickness dpml))))
(define-param resolution 10)
(define-param fcen 0.6452)
(define-param df 1)
(set! sources (list
(make source
(src (make gaussian-src (frequency fcen) (fwidth df)))
(component Ez) (center (+ r -0.1) 0 0))))
(run-sources+ 200
(at-beginning output-epsilon)
(after-sources (harminv Ez (vector3 (+ r -0.1)) fcen df)))
(run-until (/ 1 fcen) (at-every (/ 1 fcen 20) output-efield-z))
_______________________________________________ meep-discuss mailing list [email protected] http://ab-initio.mit.edu/cgi-bin/mailman/listinfo/meep-discuss
