[petsc-users] valgrind with petscmpiexec
Hi All, I tried to use valgrind to check if the simulation is valgrind clean because I saw some random communication fails during the simulation. I tried this command-line petscmpiexec -valgrind -n 576 ../../../moose-app-oprof -i input.i -log_view -snes_view But I got the following error messages: valgrind: Unable to start up properly. Giving up. ==75586== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75586_cmdline_8c3fabf2 ==75586== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75586_cmdline_8cac2243 ==75586== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75586_cmdline_da8d30c0 ==75586== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75586_cmdline_877871f9 ==75586== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75586_cmdline_c098953e ==75586== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75586_cmdline_aa649f9f ==75586== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75586_cmdline_097498ec ==75586== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75586_cmdline_bfc534b5 ==75586== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75586_cmdline_7604c74a ==75586== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75586_cmdline_a1fd96bb ==75586== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75586_cmdline_4c8857d8 valgrind: Startup or configuration error: valgrind:Can't create client cmdline file in /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75586_cmdline_4c8857d8 valgrind: Unable to start up properly. Giving up. ==75596== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75596_cmdline_bc5492bb ==75596== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75596_cmdline_ec59a3d8 valgrind: Startup or configuration error: valgrind:Can't create client cmdline file in /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75596_cmdline_ec59a3d8 valgrind: Unable to start up properly. Giving up. ==75597== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75597_cmdline_b036bdf2 ==75597== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75597_cmdline_105acc43 ==75597== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75597_cmdline_9fb792c0 ==75597== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75597_cmdline_30602bf9 ==75597== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75597_cmdline_21eec73e ==75597== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75597_cmdline_0b53e99f ==75597== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75597_cmdline_73e31aec ==75597== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75597_cmdline_486e8eb5 ==75597== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75597_cmdline_db8c194a ==75597== VG_(mkstemp): failed to create temp file: /var/tmp/pbs.3110013.sawtoothpbs/valgrind_proc_75597_cmdline_839780bb I did a bit search online, and found something related https://stackoverflow.com/questions/13707211/what-causes-mkstemp-to-fail-when-running-many-simultaneous-valgrind-processes But do not know what is the right way to fix the issue. Thanks so much, Fande,
Re: [petsc-users] Transform of algorithm containing zgemv/FFT/Slicing into PETSc-functions
I think you can again use MatDenseGetArray() and do the copies directly respecting the shift that you desire. Each process will just do the local rows so you need not worry about parallelism. I think it may be as simple as get the array pointer for A, shift it by number of local rows * number of columns then do a PetscArraycpy() to copy the B values into the shifted location in A. Barry > On Dec 14, 2020, at 5:18 PM, Roland Richter wrote: > > Dear all, > > I am currently working on the transformation of an algorithm implemented > using armadillo into PETSc. It is a forward/backward transformation, and > boils down to the following steps (for the forward transformation): > > Assumed I have matrices A and B, defined as > > A = |aa ab ac ad| > |ae af ag ah| > |ai aj ak al| > > B = |ba bb bc| > |be bf bg| > |bi bj bk| > > with the number of rows in A and B always equal, but number of columns in B > always less or equal than half the number of columns in A (Example here is > only for demonstration, I am aware of that 3 is not smaller or equal than 2). > > Moreover, I have vectors x and y, with x defined as > > x = |xa xb xc xd| > > and y defined as > > y = |ya yb yc| > > The number of elements in x corresponds to the number of columns in A, and > the number of elements y accordingly correspond to the number of columns in B. > > Now, the transformation can be described as > > Set all values in A to zero > Copy B into A with an offset of a0: > A(a0 = 1) = |0 ba bb bc| > |0 be bf bg| > |0 bi bj bk| > Multiply every row in A elementwise with y, including offset, resulting in > A(a0 = 1) = |0 ba*ya bb*yb bc*yc| > |0 be*ya bf*yb bg*yc| > |0 bi*ya bj*yb bk*yc| > Apply a 1d-FFT over each row of A, resulting in A' > Multiply every row in A' elementwise with x, resulting in > A'(a0 = 1) = |aa'*xa (ba*ya)'*xb (bb*yb)'*xc (bc*yc)'*xd| > |ae'*xa (be*ya)'*xb (bf*yb)'*xc (bg*yc)'*xd| > |ai'*xa (bi*ya)'*xb (bj*yb)'*xc (bk*yc)'*xd| > Based on earlier questions, I already know how to apply a vector to each row > of a matrix (by using .diag()) and how to apply an FFT over each row of a > distributed matrix by using FFTW. Still, I am not aware of a method for > copying B into A with an offset, and therefore I would have to iterate over > each row for the copy process, which might slow down the process. Therefore, > is there a way I could make this process more efficient using the built-in > functions in PETSc? Unfortunately, I am not that familiar with all the > functions yet. > > Thanks! > > Roland >
[petsc-users] Transform of algorithm containing zgemv/FFT/Slicing into PETSc-functions
Dear all, I am currently working on the transformation of an algorithm implemented using armadillo into PETSc. It is a forward/backward transformation, and boils down to the following steps (for the forward transformation): Assumed I have matrices A and B, defined as A = |aa ab ac ad| |ae af ag ah| |ai aj ak al| B = |ba bb bc| |be bf bg| |bi bj bk| with the number of rows in A and B always equal, but number of columns in B always less or equal than half the number of columns in A (Example here is only for demonstration, I am aware of that 3 is not smaller or equal than 2). Moreover, I have vectors x and y, with x defined as x = |xa xb xc xd| and y defined as y = |ya yb yc| The number of elements in x corresponds to the number of columns in A, and the number of elements y accordingly correspond to the number of columns in B. Now, the transformation can be described as * Set all values in A to zero * Copy B into A with an offset of a0: o A(a0 = 1) = |0 ba bb bc| |0 be bf bg| |0 bi bj bk| * Multiply every row in A elementwise with y, including offset, resulting in o A(a0 = 1) = |0 ba*ya bb*yb bc*yc| |0 be*ya bf*yb bg*yc| |0 bi*ya bj*yb bk*yc| * Apply a 1d-FFT over each row of A, resulting in A' * Multiply every row in A' elementwise with x, resulting in o A'(a0 = 1) = |aa'*xa (ba*ya)'*xb (bb*yb)'*xc (bc*yc)'*xd| |ae'*xa (be*ya)'*xb (bf*yb)'*xc (bg*yc)'*xd| |ai'*xa (bi*ya)'*xb (bj*yb)'*xc (bk*yc)'*xd| Based on earlier questions, I already know how to apply a vector to each row of a matrix (by using .diag()) and how to apply an FFT over each row of a distributed matrix by using FFTW. Still, I am not aware of a method for copying B into A with an offset, and therefore I would have to iterate over each row for the copy process, which might slow down the process. Therefore, is there a way I could make this process more efficient using the built-in functions in PETSc? Unfortunately, I am not that familiar with all the functions yet. Thanks! Roland
